Home Depot - Product Search Relevance

Kaggle Competition

Shoppers rely on Home Depot’s product authority to find and buy the latest products and to get timely solutions to their home improvement needs. From installing a new ceiling fan to remodeling an entire kitchen, with the click of a mouse or tap of the screen, customers expect the correct results to their queries – quickly. Speed, accuracy and delivering a frictionless customer experience are essential.

In this competition, Home Depot is asking Kagglers to help them improve their customers' shopping experience by developing a model that can accurately predict the relevance of search results.

Search relevancy is an implicit measure Home Depot uses to gauge how quickly they can get customers to the right products. Currently, human raters evaluate the impact of potential changes to their search algorithms, which is a slow and subjective process. By removing or minimizing human input in search relevance evaluation, Home Depot hopes to increase the number of iterations their team can perform on the current search algorithms.

This Competetion is hosted on Kaggle  | My profile at Kaggle.com -- Sampath Sree

• Date: Jan 2016 to Apr 2016
• Client: Home Depot
• Category: Text Mining
• Language: Python 3 with sklearn

Close Project

Network Intrusion Detection

KDD Competition 1999

This makes sense on today’s smarter planet, where instrumented, interconnected and intelligent businesses collect, process, use and store increasing amounts of data. Adversaries are exploiting zero-day vulnerabilities, taking advantage of risks introduced by cloud and mobile computing, and applying social engineering tactics to compromise user accounts. Advanced attackers are both patient and clever, evading detection at the network level.

The task for the classifier learning contest organized in conjunction with the KDD'99 conference was to learn a predictive model (i.e. a classifier) capable of distinguishing between legitimate and illegitimate connections in a computer network.

As we see this can be a zero day vulnerability, we used spark which is a fast and general engine for large-scale data processing. This would help in getting real time analysis of the network events. We have implemented using Pyspark.

Competition Details - KDD '99 Competition

For more technical details - Project Home Page

Close Project

Google Page Rank

Calculation of Page rank for each page in wikipedia data set

PageRank is an algorithm used by Google Search to rank websites in their search engine results. PageRank was named after Larry Page, one of the founders of Google. PageRank is a way of measuring the importance of website pages. According to Google:

PageRank works by counting the number and quality of links to a page to determine a rough estimate of how important the website is. The underlying assumption is that more important websites are likely to receive more links from other websites. - From Wikipedia

In this project, we have implemented a small search engine on wikipedia data set by calculating page ranks and established inverted indices. For a given search term we display the results based on inverted indices followed by page ranks.

For technical implementation - Github

Student Information System

Future Niners

This project is main purpose is to display content from database which is designed as a part of curriculum for Database systems. This is developed in PHP and MySQL as backend system.

1. Session maintenance
2. Login
3. Registration
4. Admission applications
5. Displaying details of College

Source Code: Github

Title: Programmer Analyst & ETL Developer

Location: Chennai, INDIA

Time Period: June 2013 to Jan 2015

Description:

• Designing and developing of data provisioning methods using INFORMATICA Power Center
• Administered and Configured Informatica Server for Cognizant TDM (Test Data Management) Center of Excellence team
• Provided training for associates on Informatica TDM over Informatica server cluster
• Developed auxiliary applications for Informatica modules - Java, Selenium
• Designed and Developed workflows for TDM for each test cycle at CBEYOND Client
• Designed and developed a proof of concept for EYE MED Health Care Client in Test Data management which turned into a full scale project
• Developed a Framework "Semi-Structured Data Protector" (SSDP) - Health care framework for masking PII, PHI & PCI information in HL7, ADT & EDI file
• Achieved hands-on expertise with IBM OPTIM, INFORMATICA SERVER ADMINISTRATION, Power Center, Power Exchange, ILM TDM, B2B Data Transformation and Data Validation Option
• Awarded as Pillar of the month for delivering best solutions for clients in Database modelling and Data Management

Back to Time line

Title: Big Data Engineer

Location: Atlanta, GA

Time Period: July 2016 to Present

Description:

• Developing Big Data solutions using Apache Spark, Cassandra(DSE), Kafka, Hadoop and Elastic Search
• A member of exploratory team, tasked to evaluate Other NoSQL Databases like DSE Cassandra to replace current homegrown NoSQL database
• Key Player in Designing Cassandra Data Model and Complete pipeline flow of the system to ensure resiliency of the system
• Developed Spark Batch and Spark Streaming pipelines in Scala, which helped in moving supplier files from NFS to Cassandra, Elastic Search and HDFS while transforming flat structured files to AVRO Format
• Automated installation and configuration of Cassandra, Spark and Hadoop using shell scripts
• Increased performance of complete pipeline by collectively tuning Spark, Cassandra and Elastic Search based on Cluster Infrastructure and workload size
• Implemented custom Serialization wrapper for Avro Generated classes to work with Spark Kyro Serializer
• Implemented generic Hadoop, Kafka and Spark Utility libraries to be used across teams, so that we don’t reinvent the wheel
• Stood 4th in Hackathon of year 2017 among 30 teams in Equifax globally, with idea of using Spark ML to detect and alert Supplier of any fraudulent activities for given list of Individuals of their Credit Activity
• Increased the data transparency in hierarchical fashion through generating Metrics about data, by implementing a Scala based generic Statistics library for AVRO Format, which takes in configuration in a conventional way for users to easily configure
• Implemented Avro Field level encryption using both Format-Preserving Encryption(FPE) and Protegrity
• Designed and developing a Spring boot application for the Users to orchestrate jobs. Behind scenes these are spark jobs, this App handles the issues of the systems in the zoo
• Promoted within 8 months of joining, due to exceptional performance and eagerness to learn

Back to Time line