Hi,
I'm a software Engineer with hands-on experience in designing and executing solutions for complex business problems involving large-scale data warehousing, real-time analytics and reporting solutions. Supportive and enthusiastic team player dedicated to streamlining processes and able to analyze code and engineer well-researched, cost-effective and responsive solutions.
I 'm working in a multinational and leading firm in big data technologies in Pakistan. I am working as a team Lead in data ingestion and Transformation to build a Big Data Platform since 2014. We are building a platform that helps enterprises overcome the business and technological complexity of Big Data. The platform is built on the existing Big Data ecosystem by integrating diverse set of technologies, such as Apache Spark, Hadoop, Solr, etc., into a unified platform. It addresses the needs of both business users and developers through a user-friendly graphics interface for easy configuration and a set of APIs for developing the Big Data Application. My Responsibilities include following areas
Data Ingestion
I am responsible for data ingestion from different sources like file systems and databases, web-services and messaging queues. Experience designing and implementing fast and efficient data acquisition using Big Data processing techniques and tools
Tools: Spark, Spark Streaming, MYSQL, SQL Server, Hadoop Ecosystem, AWS S3, Kinesis, RedShift, Apache Kafka, Flume, Cassandra, Scala, Java, Eclipse,