Project hadooptrabajos

Actualmente en CIBER nos encontramos en plena expansión a nivel nacional e internacional por lo que necesitamos incorporar en nuestra plantilla a profesionales con experiencia en el sector IT y motivados por trabajar en el sector bancario. Si tienes experiencia como programador, analista, jefe de proyecto o director en proyectos vinculados con Business Intell...de formación a nivel técnico y en capacidades de consultoría  ·Retribución salarial acorde con la valía y experiencia. Tenemos clientes en todo el mundo y en todas las industrias, por lo que podrás aprovechar tu especialización para adquirir experiencia y desarrollar habilidades que añadan valor a tu carrera en las siguientes tecnologías: - Business O...

...curriculum y nos pondremos en contacto contigo. Los requisitos mínimos que buscamos son los siguientes: Que seas ingeniero técnico o superior en informática Entre 1 y 2 años de experiencia en Java/J2EE Que tengas experiencia en aplicaciones Web Que tengas conocimientos en bases de datos NoSQL (MongoDB, Neo4J, Cassandra...) aunque no tengas práctica laboral. Y que conozcas BigData (Spark, Storm, Hadoop...) aunque todavía no hayas tenido la ocasión de ponerlo en práctica. Además vamos a valorar los siguientes requisitos si los tienes: Que tengas conocimiento, práctica o ganas de trabajar bajo metodologías ágiles Que poseas conocimiento y/o práctica laboral en: Análisis de datos, Python...

S21sec, líder en proyectos de Ciberseguridad con más de 13 años de experiencia avalada y dedicación exclusiva a la seguridad con 300 expertos en ciberseguridad reconocidos experiencia y conocimiento. El 20 % de las compañías del Dow Jones Eurostoxx 50 son clientes de S21sec. En el mercado financiero, analistas inter...PythonExperiencia en diseño de aplicaciones altamente escalables y en entornos de virtualización (VMWare ESXi, Xen) y containers ligeros (Docker)Experiencia en diseño y administración de BB.DD. relacionales (PostgreSQL, MySQL)Experiencia en el tratamiento y gestión de grandes cantidades de información tanto en sistemas de BB.DD. relacionales (MySQL y PostgreSQL) como en sistemas NoSQL (Ha...

Desde Sotec Consulting y para importante cliente del sector tecnológico, precisamos incorporar un desarrollador BIG DATA para desarrollo y mantenimiento de procesos BIGDATA/:-Experiencia en DESARROLLO/IMPLANTACIÓN/MANTENIMIENTO de entornos Hadoop/BigData (HDFS, HIVE, PIG, TEZ, SQOOP, ETC)-Experiencia en SQL-Experiencia en entorno Unix (shellscripts)-Deseable experiencia en lenguaje Java, Talend, Datawarehouse, SAP-BIUbicación: AlcobendasHorario: 09.00 - 18.00-

ALTEN precisa incorporar a su equipo un Devops para trabajar en un importante cliente del campo de la investigación científica. Empresa del sector bancario. Que tenga experiencia en: Estabilizar, mantener o instalar entornos o paquetes. Soporte a equipos de Desarrollo y Calidad. UNIX Linux Red Hat ORACLE Jenkins, Maven Elastic Search, Hadoop Bash Scripting Servidores / Datacenters Imprescindible nivel alto de inglés Se valora: Conocimientos en Java y Oracle (desarrollo y BBDD) ALTEN te ofrece: Incorporación a una empresa multinacional competitiva en continuo crecimiento. Integración en un equipo de profesionales altamente cualificado, con un buen clima laboral, innovador y dinámico. Formación especializada y desarrollo profesional con...

...Resolución de incidencias Realización de pruebas. Realización de documentación técnica Configuración de entornos de desarrollo Requisitos: Conocimientos de desarrollo Java, J2EE, HTML, css, xml, spring, hibernate, jdbc Conocimientos de BBDD: Oracles, access, MySQL, SQL server Conocimientos de sistemas operativos Unix y Windows Capacidad de análisis Requisitos deseados Conocimientos de Big Data: Hadoop, spark, cloudera, map reduce, mongo, pig..etc Procesamiento distribuido Metodologías ágiles y scrum M2C Ofrece: Plan de Carrera Salario Competitivo Beneficios Sociales Horario Flexible Formación y Certificaciones Lugar de trabajo: Madrid Imprescindible residir en la provincia del puesto vacante Analista-Programado...

...el análisis de módulos de la aplicación y pruebas unitarias.<br />Aplicación en el ámbito del Ciclo de Vida de producto software y Gestión de Cambios de producto software.<br />Experiencia en análisis y gestión de requerimientos.<br /><br />Conocimientos de modelizado y análisis:<br /><br /><br />UML<br />Life Cycle Management<br /><br />Ingeniería de Software<br />Experiencia en el ecosistema Hadoop y Big Data.<br />- Amplia experiencia en Business Intelligence, Data Warehouse y BBDD<br />- Optimización de procesos/flujos de información,<br />- Análisis de soluciones BI<br />Life Cycle Management<b...

BluePatagon empresa líder en tecnologías de Business Intelligence & Business Analytics, está en búsqueda de Especialista en Big Data (Hadoop - Hortonworks), para importante cliente de CABA. Experiencia: mínima de 1 año en tecnologías Hadoop: Experiencia en desarrollo de aplicaciones (MapReduce + HDFS). Big Data: Familiaridad con el ecosistema (Hive, Pig, HBase, etc). y conceptos de escalabilidad, análisis en tiempo real, procesamiento de datos distribuidos. Linux: Uso Avanzado (manejo de servicios de SO, administración, shell scripting, seguridad) Programacion: POO (Java preferentemente, Python) Base de datos: RDBMS (Oracle, MySQL, PostgreSQL), NoSQL (HBase, Cassandra) Data Exchange y configura...

.../>Conocimientos imprescindible:<br />Conocimientos de arquitectura de sistemas<br />Sizing<br />Clustering<br />Administración de servidores valorables:<br />Conocimientos de Administración de Server<br />Scripting<br />Extensiones<br />Publisher y/o NprintingSe valora:<br />Certificaciones en Qlikview (Designer/Developer/System administrator)<br />Experiencia en entornos Big data (Hadoop, Hortonworks, Cloudera, Hive, Hapreduce)...

Hacer 4 instalaciones Linux BDs Cloudera/Hadoop/Impala Desarrollar 3 programa batch en Linux para carga de registros, leer desde archivos en fmto textoCSV y cargarlos al BigData. Cada archivo textoCSV tiene 500 registros y cada registro tiene 278 campos. Implementar una herramienta de consultas sobre los registros del BigData. Todo desarrollado sobre herramientas opensource sobre Linux.

Por favor, regístrate o inicia sesión para ver los detalles.

El backend estaría compuesto por el lenguaje de programación Python, aunque también se programaría en Javascript haciendo uso de tecnologías como NodeJS.  La base de datos sería por un lado relacional PosgreSQL y por otro lado también utili...una no relacional tipo MongoDB Para la plataforma en sí haríamos uso de tecnologías de frontend como AngularJS, HTML5, CSS3 etc. Básicamente es: lenguaje de programación PYTHON en el backend (en los servidores con los procesos de análisis de información) y JAVASCRIPT con el framework de AngularJS en la parte de cliente. Para el análisis de datos se utilizarán tecnologías usadas en Big Data como Hadoop, MapReduce, Spark, ...

Persona experta en: Hadoop para el almacenamiento de interfases, PIG para el procesamiento de la alerta  y Cloudera para la administración y monitoreo de procesos. Que viva en México (de preferencia DF) para trabajar durante dos meses aproximadamente, al menos 1 tiempo completo en sitio. Se negociaría el 2do mes.

Desarrollos para gran empresa basados en: • Apache Cassandra • Apache Hadoop • Bases de datos Columnares • PowerCenter - Informatica (preferiblemente 9.X) Habilidades: -Conocimientos en las tecnologías anteriores -Trabajo en equipo

Vacantes Data4 Python + Django Web Developers Data4 esta buscando desarrolladores en Django/Python para el departamento de Data Driven Web Platforms. El departamento esta encargado de elaborar complejas plataformas usando Django, PostgreSQL+ PostGIS / Mysql, Memcached, Haystack con Solr, y su integración a Big Data con Hadoop, Hbase, R y las librerías en Python para machine learning como PyBrain, NumPy, SciPY. Trabajamos bajo la metodología ágil SCRUM con mucha apertura a la experimentación y el auto aprendizaje. Nuestros desarrolladores trabajan desde la arquitectura de la base de datos hasta la construcción de excelentes UI y UX de las plataformas. Data4 tiene su base en la Ciudad de México.  Responsabilidades<...

...Gran experiencia en NoSQL /HBASE, Cassandra o similares, Neo4j). Voluntad de aprender e implementar nuevas tecnologías BigData, según sea necesario. Iniciativa y capacidad de trabajar de manera independiente y en equipo. Experiencia con Sotrm en soluciones real time analytics. Experiencia en procesamiento paralelo (MPI, OpenMP) como ventaja competitiva para el puesto. Experto en comprensión en Hadoop HDFS y MapReduce. Pensamiento creativo (Out of the box) Capacidad en gestión de equipos <em>InnoQuant acaba de ser seleccionado como uno de los 10 startups tecnológicas más prometedoras en España.  Somos un equipo experimentado de profesionales de TI que trabajan en tiempo real plataforma de análisis de grandes datos...

Se precisa programador JavaScript para crear una extensión multi-navegador (Chrome, Firefox, Safari, IE) que se ocupará de salvaguardar la privacidad de los usuarios que se la instalen. Se requiere: * Amplia experiencia desarrollando con JavaScript (patrones de diseño, buenas prácticas, uso de prototi...la librería jQuery * Conocimiento de alguna tecnología web de servidor (java, php, python, ruby…) para gestionar altas de usuarios y hacer un registro muy simple de cierta actividad. * Inglés avanzado Se valorará: * Conocimientos de HTML5 y CSS3 * Experiencia portando extensiones a otros navegadores: , Kango, … * Diseño y maquetación web * Experiencia con motores de búsqueda de Big Data: Ha...

... El web-app sera construido con un solo usuario, por lo que no es necesario preocuparse en crear todas las cuentas. Sera escalable a mas usuarios en base a la demanda (AWS). No es necesario realizar las applicaciones de Android/iOS para mobile, excepto preparar el Web service para que pueda interactuar con ellas en un futuro cercano. 1 - Configuracion del Cloud: Amazon EC2 con MongoDB (or Hadoop) e instalar OpenCV. Configurar la micro-instancia en Ubuntu con Eclipse como IDE. (3 dias) 2 - Implementar una Service Component Library : la libreria usara OpenCV para realizar operaciones en C++ en AWS EC2. No es neceserio codificar OpenCV, simplemente dejarlo listo para futuras implementaciones. (1 Week) 3 - Implementar un AWS web service (CloudFront?), “Receiver”:...

...usuario. El web-app sera construido con un solo usuario, por lo que no es necesario preocuparse en crear todas las cuentas. Sera escalable a mas usuarios en base a la demanda (AWS). No es necesario realizar las applicaciones de Android/iOS para mobile, excepto preparar el Web service para que pueda interactuar con ellas en un futuro cercano. 1 - Configuracion del Cloud: Amazon EC2 con MongoDB (or Hadoop) e instalar OpenCV. Configurar la micro-instancia en Ubuntu con Eclipse como IDE. (3 dias) 2 - Implementar una Service Component Library : la libreria usara OpenCV para realizar operaciones en C++ en AWS EC2. No es neceserio codificar OpenCV, simplemente dejarlo listo para futuras implementaciones. (1 Week) 3 - Implementar un AWS web service (CloudFront?), “Receiver”:...

Se precisa programador JavaScript para crear una extensión multi-navegador (Chrome, Firefox, Safari, IE) que se ocupará de salvaguardar la privacidad de los usuarios que se la instalen. Dicha extensión se desarrollaría en modo open source a partir de la ya existente Se requiere: * Amplia ...librería jQuery * Conocimiento de alguna tecnología web de servidor (java, php, python, ruby…) para gestionar altas de usuarios y hacer un registro muy simple de cierta actividad. * Inglés avanzado Se valorará: * Conocimientos de HTML5 y CSS3 * Experiencia portando extensiones a otros navegadores: , Kango, … * Diseño y maquetación web * Experiencia con motores de búsqueda de Big Data: Hadoop, ...

I have a Ubuntu 22.04 server, currently with spark running perfectly. I want, however, integrate spark with Hive warehouse. In this project, you're required to perform the installing of Hadoop and hive and, after that, update settings in spark so it can connect with hive warehouse properly. You'll be provided with remote access via TeamViewer or anydesk.

Looking for an android developer Java,Kotlin,JSP, Servlets, MQ series, Hibernate, Python, Microservices, Swings, WebSphere-Portals, HTML5, Bootstrap, JavaScript, Bigdata, Linux Frameworks Android Mobile App Development, Spring boot, Struts, Hadoop This will be ongoing

...Twitter data using a visually appealing interface. - **Real-time Data Updates**: The dashboard should be capable of updating in real-time to reflect any changes in the forecast. The ideal freelancer for this project should have proven experience in time series forecasting, Python programming, and developing interactive dashboards. A solid grasp of data visualization principles would be a plus. The output should be in a form of a python file with 4 models above and related dynamic visualizations. The text preprocessing is required and can be done in any necessary technologies (HADOOP/SPARK,NOSQL/SQL databases) - screenshots of the bash code are required....

...data engineers who can tackle essential tasks within the domain. The ideal candidate for this project should be able to: - Perform data analysis: This includes understanding, interpreting, and presenting data to inform decision-making. - Manage data integration: The candidate should be capable of combining data from various sources to create a cohesive dataset. - Develop data visualization: The engineers will need to create meaningful and understandable visual representations of the analyzed data. The necessary skills for this role are: - Proficient in Python: The candidate should be well-versed in Python programming language to work with our system. - Knowledge of Hadoop: Experience with Hadoop is a must for this position to handle the data integration and data ...

Seeking a skilled developer to optimize and enhance the architecture of our existing web scraper application. The application is currently built using NestJS and PostgreDB, and we are looking to scale it up and leverage cloud functionality for improved p...error handling, rate limiting, and IP rotation. - Strong problem-solving skills and ability to optimize application performance. - Excellent communication and collaboration skills. Nice to have: - Experience with PostgreDB and database optimization techniques. - Knowledge of additional programming languages like Python or Java. - Familiarity with data processing frameworks like Apache Spark or Hadoop. - Experience with data visualization and reporting tools. Potential for ongoing collaboration based on performance and future req...

I need someone to create the dag and trigger it I am working on a migration project from hadoop to bigquery....more details will be shared via chat

I am seeking a skilled data engineering trainer, speed in using Hadoop, Apache Spark, and SQL is paramount. Your expertise will guide me through nuanced uses of these technologies, with a particular focus on data migration. Key Requirements: - Proficiency in Hadoop, Apache Spark, and SQL - More than 10 hours availability weekly - Proven experience in real-world data migration projects Ideal candidates should have a flair for explaining complex concepts in simple language. This engagement will focus on moving data from diverse sources into a data warehouse, thereby making it readily available for business intelligence functions.

I'm in need of a proficient professional versed in Java Hadoop cluster. Please place your bids immediately. $20 for this project

I am in urgent need of Hadoop/Spark developer who is proficient in both Scala and Python for a data processing task. I have a huge volume of unstructured data that needs to be processed and analyzed swiftly and accurately. Key Project Responsibilities: - Scrubbing and cleaning the unstructured data to detect and correct errors. - Designing algorithms using Scala and Python to process data in Hadoop/Spark. - Ensuring effective data processing and overall system performance. The perfect fit for this role is a professional who has: - Expertise in Hadoop and Spark frameworks. - Proven experience in processing unstructured data. - Proficient coding skills in both Scala and Python. - Deep understanding of data structures and algorithms. - Familiarity wit...

...and System Analyst About Luxetrip: Luxetrip is a groundbreaking luxury travel startup that aims to revolutionize the high-end travel experience by leveraging artificial intelligence, personalization, and exclusive partnerships. We are seeking a talented and experienced freelance AI architect and system analyst to help us design, develop, and launch our innovative online travel platform. Project Overview: The selected freelancer will be responsible for architecting and building the core AI-powered systems and algorithms that drive Luxetrip's online travel business. This will involve close collaboration with our leadership team, domain experts, and technology partners to define the platform's key features, functionalities, and performance requirements. The freelan...

We are looking for an Informatica BDM developer with 7+ yrs of experience, who can support us for 8 hours in a day from Mon - Friday. Title : Informatica BDM Developer Experience : 5 + Yrs 100%Remote Contract : Long term Timings: 10:30 am - 07:30 pm IST Required Skills: Informatica Data Engineering, DIS and MAS • Databricks, Hadoop • Relational SQL and NoSQL databases, including some of the following: Azure Synapse/SQL DW and SQL Database, SQL Server and Oracle • Core cloud services from at least one of the major providers in the market (Azure, AWS, Google) • Agile Methodologies, such as SCRUM • Task tracking tools, such as TFS and JIRA

...which will include parameters such as patient age ranges, geographical regions, social conditions, and specific types of cardiovascular diseases. Key responsibilities: - Process distributed data using Hadoop/MapReduce or Apache Spark - Developing an RNN model (preferably Python) - Analyzing the complex CSV data (5000+ records) - Identifying and predicting future trends based on age, region, types of diseases and other factors - Properly visualizing results in digestible diagrams Ideal candidates should have: - Experience in data analysis with Python - Solid understanding of Hadoop/MapReduce or Apache Spark - Proven ability in working with Recurrent Neural Networks - Excellent visualization skills to represent complex data in static or dynamic dashboards - Experien...

I am looking for an experienced Senior Data Engineer for interview training. Your primary responsibility would be data cleaning and preprocessing, design and optimize database and implement ETL processes. Key responsibilities include: - Clean and preprocess data to ensure its quality and ef...Clean and preprocess data to ensure its quality and efficiency. - Design and optimize databases, aiming for both flexibility and speed. - Implement ETL (Extract, Transform, Load) processes to facilitate the effective and secure moving of data. Skills and Experience: - Proficient in Python, SQL, and Hadoop. - Expertise in handling medium-sized databases (1GB-1TB). - Proven track record in ETL processes handling. Your expertise in these areas will be crucial to the successful completion of thi...

I have encountered a problem with my Hadoop project and need assistance. My system is showing ": HADOOP_HOME and are unset", and I am not certain if I've set the HADOOP_HOME and variables correctly. This happens creating a pipeline release in devops. In this project, I am looking for someone who: - Has extensive knowledge about Hadoop and its environment variables - Can determine whether I have set the HADOOP_HOME and variables correctly and resolve any issues regarding the same - Able to figure out the version of Hadoop installed on my system and solve compatibility issues if any I will pay for the solution immediately.

*Title: Freelance Data Engineer* *Description:* We are seeking a talented freelance data engineer to join our team on a project basis. The ideal candidate will have a strong background in data engineering, with expertise in designing, implementing, and maintaining data pipelines and infrastructure. You will work closely with our data scientists and analysts to ensure the smooth flow of data from various sources to our data warehouse, and to support the development of analytics and machine learning solutions. This is a remote position with flexible hours. *Responsibilities:* - Design, build, and maintain scalable and efficient data pipelines to collect, process, and store large volumes of data from diverse sources. - Collaborate with data scientists and analysts to understand data ...

As a beginner, I am seeking a knowledgeable developer who can guide me on effectively using Google Cloud for Hadoop, Spark, Hive, pig, and MR. The main goal is data processing and analysis. Key Knowledge Areas Needed: - Google Cloud usage for big data management - Relevant functionalities of Hadoop, Spark, Hive, pig, and MR - Best practices for data storage, retrieval, and workflow streamlining Ideal Skills: - Extensive Google Cloud experience - Proficiency in Hadoop, Spark, Hive, Pig, and MR for data processes - Strong teaching abilities for beginners - Demonstrated experience in data processing and analysis.

As a beginner, I am seeking a knowledgeable developer who can guide me on effectively using Google Cloud for Hadoop, Spark, Hive, pig, and MR. The main goal is data processing and analysis. Key Knowledge Areas Needed: - Google Cloud usage for big data management - Relevant functionalities of Hadoop, Spark, Hive, pig, and MR - Best practices for data storage, retrieval, and workflow streamlining Ideal Skills: - Extensive Google Cloud experience - Proficiency in Hadoop, Spark, Hive, Pig, and MR for data processes - Strong teaching abilities for beginners - Demonstrated experience in data processing and analysis.

As a beginner, I am seeking a knowledgeable developer who can guide me on effectively using Google Cloud for Hadoop, Spark, Hive, pig, and MR. The main goal is data processing and analysis. Key Knowledge Areas Needed: - Google Cloud usage for big data management - Relevant functionalities of Hadoop, Spark, Hive, pig, and MR - Best practices for data storage, retrieval, and workflow streamlining Ideal Skills: - Extensive Google Cloud experience - Proficiency in Hadoop, Spark, Hive, Pig, and MR for data processes - Strong teaching abilities for beginners - Demonstrated experience in data processing and analysis.

...commonly used packages specially with GCP. Hands on experience on Data migration and data processing on the Google Cloud stack, specifically: Big Query Cloud Dataflow Cloud DataProc Cloud Storage Cloud DataPrep Cloud PubSub Cloud Composer & Airflow Experience designing and deploying large scale distributed data processing systems with few technologies such as PostgreSQL or equivalent databases, SQL, Hadoop, Spark, Tableau Hands on Experience with Python-Json nested data operation. Exposure or Knowledge of API design, REST including versioning, isolation, and micro-services. Proven ability to define and build architecturally sound solution designs. Demonstrated ability to rapidly build relationships with key stakeholders. Experience of automated unit testing, automated integra...

I am looking for a skilled professional who can efficiently set up an big data ...professional who can efficiently set up an big data cluster. REQUIREMENTS: • Proficiency in Elasticsearch,Hadoop,Spark,Cassandra • Experience in working with large-scale data storage (10+ terabytes). • Able to structure data effectively. SPECIFIC TASKS INCLUDE: - Setting up the Elasticsearch,Hadoop,Spark,Cassandra big data cluster. - Ensuring the data to be stored is structured. - Prep for the ability to handle more than 10 terabytes of data. The ideal candidate will have substantial experience in large data structures and a deep understanding of the bigdata database technology. I encourage experts in big data management and those well-versed with the best practices of bigdat...

We are looking for an Informatica BDM developer with 7+ yrs of experience, who can support us for 8 hours in a day from Mon - Friday. Title : Informatica BDM Developer Experience : 5 + Yrs 100%Remote Contract : Long term Timings: 10:30 am - 07:30 pm IST Required Skills: Informatica Data Engineering, DIS and MAS • Databricks, Hadoop • Relational SQL and NoSQL databases, including some of the following: Azure Synapse/SQL DW and SQL Database, SQL Server and Oracle • Core cloud services from at least one of the major providers in the market (Azure, AWS, Google) • Agile Methodologies, such as SCRUM • Task tracking tools, such as TFS and JIRA

I am seeking a skilled professional proficient in managing big data tasks with Hadoop, Hive, and PySpark. The primary aim of this project involves processing and analyzing structured data. Key Tasks: - Implementing Hadoop, Hive, and PySpark for my project to analyze large volumes of structured data. - Use Hive and PySpark for sophisticated data analysis and processing techniques. Ideal Skills: - Proficiency in Hadoop ecosystem - Experience with Hive and PySpark - Strong background in working with structured data - Expertise in big data processing and data analysis - Excellent problem-solving and communication skills Deliverables: - Converting raw data into useful information using Hive and Visualizing the results of queries into the graphical represe...

...R), and other BI essentials, join us for global projects. What We're Looking For: Business Intelligence Experts with Training Skills: Data analysis, visualization, and SQL Programming (Python, R) Business acumen and problem-solving Effective communication and domain expertise Data warehousing and modeling ETL processes and OLAP Statistical analysis and machine learning Big data technologies (Hadoop, Spark) Agile methodologies and data-driven decision-making Cloud technologies (AWS, Azure) and data security NoSQL databases and web scraping Natural Language Processing (NLP) and sentiment analysis API integration and data architecture Why Work With Us: Global Opportunities: Collaborate worldwide across diverse industries. Impactful Work: Empower businesses through data-drive...

I'm launching an extensive project that needs a proficient expert in Google Cloud Platform (including BigQuery, GCS, Airflow/Composer), Hadoop, Java, Python, and Splunk. The selected candidate should display exemplary skills in these tools, and offer long-term support. Key Responsibilities: - Data analysis and reporting - Application development - Log monitoring and analysis Skills Requirements: - Google Cloud Platform (BigQuery, GCS, Airflow/Composer) - Hadoop - Java - Python - Splunk The data size is unknown at the moment, but proficiency in managing large datasets will be advantageous. Please place your bid taking into account all these factors. Your prior experience handling similar projects will be a plus. I look forward to working with a dedicated a...

...commonly used packages specially with GCP. Hands on experience on Data migration and data processing on the Google Cloud stack, specifically: Big Query Cloud Dataflow Cloud DataProc Cloud Storage Cloud DataPrep Cloud PubSub Cloud Composer & Airflow Experience designing and deploying large scale distributed data processing systems with few technologies such as PostgreSQL or equivalent databases, SQL, Hadoop, Spark, Tableau Hands on Experience with Python-Json nested data operation. Exposure or Knowledge of API design, REST including versioning, isolation, and micro-services. Proven ability to define and build architecturally sound solution designs. Demonstrated ability to rapidly build relationships with key stakeholders. Experience of automated unit testing, automated integra...

As an ecommerce platform looking to optimize our data management, I require assistance with several key aspects of my AWS big d...an ecommerce platform looking to optimize our data management, I require assistance with several key aspects of my AWS big data project, including: - Data lake setup and configuration - Development of AWS Glue jobs - Deployment of Hadoop and Spark clusters - Kafka data streaming The freelancer hired for this project must possess expertise in AWS, Kafka, and Hadoop. Strong experience with AWS Glue is essential given the heavy utilization planned for the tool throughout the project. Your suggestions and recommendations regarding these tools and technologies will be heartily welcomed, but keep in mind specific tools are needed ...

...Queries: Write a SQL query to find the second highest salary. Design a database schema for a given problem statement. Optimize a given SQL query. Solution Design: Design a parking lot system using object-oriented principles. Propose a data model for an e-commerce platform. Outline an approach to scale a given algorithm for large datasets. Big Data Technologies (if applicable): Basic questions on Hadoop, Spark, or other big data tools. How to handle large datasets efficiently. Writing map-reduce jobs (if relevant to the role). Statistical Analysis and Data Processing: Write a program to calculate statistical measures like mean, median, mode. Implement data normalization or standardization techniques. Process and analyze large datasets using Python libraries like Pandas. Rememb...

...customer-centric software products · Analyze existing software implementations to identify areas of improvement and provide deadline estimates for implementing new features · Develop software applications using technologies that include and not limited to core Java (11+ ), Kafka or messaging system, Web Frameworks like Struts / Spring, relational (Oracle) and non-relational databases (SQL, MongoDB, Hadoop, etc), with RESTful microservice architecture · Implement security and data protection features · Update and maintain documentation for team processes, best practices, and software runbooks · Collaborating with git in a multi-developer team · Appreciation for clean and well documented code · Contribution to database design ...

Project Title: Advanced Hadoop Administrator Description: - We are seeking an advanced Hadoop administrator for an inhouse Hadoop setup project. - The ideal candidate should have extensive experience and expertise in Hadoop administration. - The main tasks of the Hadoop administrator will include data processing, data storage, and data analysis. - The project is expected to be completed in less than a month. - The Hadoop administrator will be responsible for ensuring the smooth functioning of the Hadoop system and optimizing its performance. - The candidate should have a deep understanding of Hadoop architecture, configuration, and troubleshooting. - Experience in managing large-scale data processing and storage ...

I am looking for a freelancer to help me with a Proof of Concept (POC) project focusing on Hadoop. Requirement: We drop a file in HDFS, which is then pushed to Spark or Kafka and it pushes final output/results into a database. Objective is to show we can handle million of records as input and put it in destination. The POC should be completed within 3-4 days and should have a simple level of complexity. Skills and experience required: - Strong knowledge and experience with Hadoop - Familiarity with HDFS and Kafka/Spark - Ability to quickly understand and implement a simple POC project - Good problem-solving skills and attention to detail