The Senior Data Engineer is responsible for overseeing junior data engineering activities and aiding in building the business data collection systems and processing pipelines. The role of the Senior Data Engineer is responsible for building and maintaining optimized and highly available data pipelines that facilitate deeper analysis and reporting by the Data and Analytics department.
The Senior Data Engineer builds data processing frameworks that handle the businesss growing database. He works with senior data science leadership as well as other Data and Analytics teams in leveraging data with reporting and scientific tools, for example, Tableau, R, and Spark. The Senior Data Engineer strives to continuously develop new and improved data engineering capabilities.
Objectives and Responsibilities of the Senior Data Engineer
Management and Strategy: The managerial role of the Senior Data Engineer is primarily for overseeing activities of the junior data engineering teams, ensuring proper execution of their duties and alignment with business vision and objectives. He provides senior-level contribution to a team that is responsible for the design, deployment, and maintenance of the businesss data platforms.
However, the Senior Data Analyst will also implement strategies directed at acquiring data and promoting the development of new insights across the business. The Senior Data Engineer owns and extends the businesss data pipeline through the collection, storage, processing, and transformation of large data-sets.
It is his duty to monitor the existing metrics, analyze data, and lead partnership with other Data and Analytics teams in an effort to identify and implement system and process improvements.
The Senior Data Engineer will additionally develop queries for ad hoc business projects, as well as ongoing reporting. In this capacity, the Senior Data Engineer builds a metadata system where all available data is maintained and cataloged. The Senior Data Engineer also plays a major role in the development of reliable data pipelines that translate raw data into powerful features and signals.
He designs, architects, implements, and supports key datasets that avail structured and timely access to actionable business insights. The Senior Data Engineer is additionally tasked with developing ETL processes that convert data into formats through a team of data analysts and dashboard charts.
Collaboration and Support: The Senior Data Engineer plays a collaborative role where he works closely with the businesss Data and Analytics teams, gathering technical requirements for exceptional data governance across the department and the business at large.
In this collaboration, the Senior Data Engineer works the data analysts, data warehousing engineers, and data scientists in finding and applying best practices within the Data and Analytics department as well as defining the businesss data requirements, which will ensure that the collected data is of a high quality and optimal for use across the department and the business at large.
The Senior Data Engineer will also work with senior data science management and departments beyond the Data and Analytics department in analyzing and understanding data sources, participating in design, and providing insights and guidance on database technology and data modeling best practices.
In this capacity, the Senior Data Engineer will further be required to draw performance reports and strategic proposals form his gathered knowledge and analyses results for senior data science leadership.
Analytics: The Senior Data Engineering plays an analytical role where he develops and manages scalable data processing platforms that he uses for exploratory data analysis and real-time analytics. It is also the role of the Senior Data Engineer to oversee, design, and develop algorithms for real-time data processing within the business and to create the frameworks that enable quick and efficient data acquisition.
In this capacity, the Senior Data Engineer retrieves and analyzes data through the use of SQL, Excel, among other data management systems. He also builds data loading services for the purpose of importing data from numerous disparate data sources, inclusive of APIs, logs, relational, and non-relational databases.
Knowledge and Opportunity: The Senior Data Engineer is tasked with the responsibility of contributing to the continual improvement of the businesss data platforms through his observations and well-researched knowledge. He keeps track of industry best practices and trends and through his acquired knowledge, takes advantage of process and system improvement opportunities.
The Senior Data Engineer performs similar duties as he deems fit for the proper execution of his duties and duties as delegated by the Head of Data Science, Director Data Science, Chief Data Officer, or the Employer.
Required Qualifications of the Senior Data Engineer
Education: The Senior Data Engineer must have a bachelors degree (masters preferred) in Computer Science, Applied Mathematics, Engineering, or any other technology related field. An equivalent of this educational requirement in working experience is also acceptable.
Experience: A candidate for this position must have had at least 5 years of working experience working in a data engineering department, preferably as a Data Engineer in a fast-paced environment and complex business setting. The candidate must have a demonstrated experience in building and maintaining reliable and scalable ETL on big data platforms as well as experience working with varied forms of data infrastructure inclusive of relational databases such as SQL, Hadoop, Spark and column-oriented databases such as Redshift, MySQL, and Vertica.
The candidate must also have had experience in data warehousing inclusive of dimensional modeling concepts and demonstrate proficiency in scripting languages, for example, Python, Perl, and so forth. A suitable candidate will also demonstrate machine learning experience and experience with big data infrastructure inclusive of MapReduce, Hive, HDFS, YARN, HBase, Oozie, etc. The candidate will additionally demonstrate substantial experience and a deep knowledge of data mining techniques, relational, and non-relational databases.
Communication Skills: Communication Skills for the Senior Data Engineer are just as important as they are for the Data Engineer, both in verbal and written form. The Senior Data Engineer oversees and manages junior data engineering teams and to ensure effective management, he must be capable of conveying information and instructions clearly down the line to the junior team.
Communication skills are also imperative for the Senior Data Engineer in his collaborative role where he will have to interact cross-functionally with non-technical departments. To enable effective collaborations, the Senior Data Engineer will have an exceptional ability to convey complex messages in a clear, simplified, and understandable manner.
He will also be required to draft reports and prepare presentations for senior data science leadership. These reports and presentation must be clear, concise, unambiguous, engaging and convincing, which will demand exceptional communication skills on the Senior Data Engineers part.
Skills: A candidate for this position will also demonstrate strong computer skills and a deep passion for analytics. The candidate for this position must possess an ability to perform complex data analyses with large data volumes. He will be an expert in SQL, Java, and have a keen understanding of data models and data warehouse concepts.
The candidate will demonstrate an ability translate algorithms provided by senior data science management and implement them in as well as strong knowledge in Linux, OS tools, and file-system level troubleshooting. The candidate must have substantial experience working with big data infrastructure tools such as Python, SQS, and Redshift. A suitable candidate will also be proficient Scala, Spark, Spark Streaming, AWS, and EMR.
The Senior Data Engineer must have certain preferable personal attributes that will make him that much more suited for the position. The Senior Data Engineer will be a result-driven individual, be passionate and a self-starter, be proactive requiring minimal supervision, be highly organized, have an ability to handle multiple tasks and meet tight deadlines, be a creative and strategic thinker, work comfortably work in a collaborative setting, work comfortably with senior departmental leadership, and demonstrate an ability to remain calm during times of uncertainty and stress, inspiring the same in his team.
The candidate must be a people person who is able to form strong, meaningful, and lasting connections with others, enabling smooth and continued collaborative relationships, earning him the trust of his juniors who will readily follow in his directives, and gaining the confidence of senior data science leadership.
|Data Scientist||Intercontinental Exchange||Atlanta, GA||Feb 20|
|Computer Vision Team Lead (m/f/x) - Robotics||TRA Robotics||Berlin, Germany||Feb 20|
|Eng 4, Prodt Dev Engineering||Comcast||Philadelphia, PA||Feb 20|
|Eng 3, Prodt Dev Engineering||Comcast||Philadelphia, PA||Feb 20|
|AI Programmer||Creative Assembly||Horsham, UK||Feb 20|
|Data Science Consultant||Accenture||San Diego, CA||Feb 20|
|Machine Learning R&D Engineer||Subsurface Consultants & Associates, LLC||Houston, TX||Feb 20|
|Data Analytics Developer||Catapult Systems||Houston, TX||Feb 20|
|Data Scientist||Hydrogen Group||Austin, TX||Feb 20|
|Jr. Data Scientist||Elev8 Hire Solutions||Atlanta, GA||Feb 20|
|Data Scientist||FCA Fiat Chrysler Automobiles||Detroit, MI||Feb 20|
|Boston-Based Summer Internship in Distributed Systems and Data Engineering||Armored Things||Boston, MA||Feb 20|
|Senior Data Engineer||BIZX, LLC / Slashdot Media / SourceForge.net||San Diego, CA||Feb 19|
|Engineering Lead||Zalando SE||Hamburg, Germany||Feb 19|
|AI and Machine Learning Engineer Consultant||Accenture||San Diego, CA||Feb 19|
|Director of Product Management||MRP||Philadelphia, PA||Feb 19|
|Senior Data Scientist, Modeling Team||Applecart||New York, NY||Feb 18|
|Machine Learning Scientist for SEMA||Amy Cell Talent||San Diego, CA||Feb 18|
|Data Scientist||GeoPhy||New York, NY||Feb 18|
|Software Engineer, Audio and Signal Processing||San Francisco, CA||Feb 17|
|Software Engineer, Audio and Signal Processing||San Bruno, CA||Feb 17|
|Research Scientist, Machine Learning and Intelligence||Pittsburgh, PA||Feb 17|
|Software Engineer, Imaging||San Francisco, CA||Feb 17|
|Software Engineer, Imaging||San Bruno, CA||Feb 17|
|Data Scientist, Engineering||San Bruno, CA||Feb 17|
|Data Scientist, Engineering||New York, NY||Feb 17|
|Machine Learning Solution Architect||Accenture||San Diego, CA||Feb 17|
|Scientific Software Developer for Data Analysis Application||De Novo Software||Glendale, CA||Feb 17|
|Data Scientist - Big Data Analytics Team||The Washington Post||Washington, DC||Feb 17|
|Eng 4, Software Dev & Engineering||Comcast||Englewood, CO||Feb 17|
|Data Scientist / Machine Learning Expert||Agoda||Khwaeng Pathum Wan, Thailand||Feb 17|
|Principal Engineers for Tokyo!||Indeed||Tokyo, Japan||Feb 17|
|Software Engineer, Audio and Signal Processing||Mountain View, CA||Feb 16|
|Software Engineer, Cloud Machine Learning||Sunnyvale, CA||Feb 16|
|Cloud AI Research Scientist||Mountain View, CA||Feb 16|
|Research Software Engineer, Google Brain||New York, NY||Feb 16|
|Software Engineer, Imaging||Sunnyvale, CA||Feb 16|
|Software Engineer, Imaging||Mountain View, CA||Feb 16|
|Enterprise Data Architect / Engineer||Critical Mix (Perrysburg, OH)||allows remote||Feb 16|
|Senior Data Scientist||Realtor.com||Richmond, BC, Canada||Feb 16|
|Director of Statistical Modeling||Burtch Works||Atlanta, GA||Feb 16|
|Senior Data Science Engineer||Comcast||Philadelphia, PA||Feb 16|
|Data Scientist||Gravity IT Resources||Miami, FL||Feb 16|
|Data Engineer/Wrangler||Signify Health||Dallas, TX||Feb 16|
|Software Engineer||Sentek Global||San Diego, CA||Feb 16|
|Senior Data Scientist||Faire||San Francisco, CA||Feb 15|
|Front End Developer (Remote OK)||Mix.com||Phoenix, AZ||Feb 15|
|Machine Learning Engineer||KELZAL (QELZAL CORPORATION)||San Diego, CA||Feb 15|
|Junior-Mid level Data Science Engineer||Biswas Information Technology Solutions||Herndon, VA||Feb 15|
|e-Discovery, Operations Coordinator||DTTS||Nashville, TN||Feb 15|
|Data Engineering and Analytics Developer (May 2019)||Armored Things||Boston, MA||Feb 15|
|Python Developer||UST Global||San Diego, CA||Feb 15|
|Python Developer||UST Global||Atlanta, GA||Feb 15|
|Senior Software Engineer, Machine Learning||SmartNews||San Francisco, CA||Feb 15|
|Data Scientist, Mid||Booz Allen Hamilton - Tagged||San Diego, CA||Feb 14|
|Software Engineer, YouTube Machine Learning||San Bruno, CA||Feb 14|
|Senior Data Scientist, Customer Insights||Expedia, Inc.||Bellevue, WA||Feb 14|
|Senior Director, Data Science||Expedia, Inc.||Bellevue, WA||Feb 14|
|Robotics Software Engineer||Moley Robotics||London, UK||Feb 14|
|Data Engineer||UiPath||Bellevue, WA||Feb 14|
|Senior Machine Learning Engineer - Recommendation Systems||WeWork||New York, NY||Feb 14|
|Sr Data Analyst||Gartner||Austin, TX||Feb 14|
|Data Scientist, Innovation||Bell Helicopter||Fort Worth, TX||Feb 14|
|Machine Learning Engineer - Industrial Deep Learning (m/f)||CeleraOne GmbH||Berlin, Deutschland||Feb 14|
|Machine Learning Engineer||Accenture||Houston, TX||Feb 14|
|Knowledge Graph Engineer Lead||Accenture||Miami, FL||Feb 14|
|Healthcare Payer Data Science Consultant||Accenture||Atlanta, GA||Feb 14|
|Machine Learning Engineer||Accenture||Dallas, TX||Feb 14|
|Knowledge Graph Engineer Manager||Accenture||Philadelphia, PA||Feb 14|
|Knowledge Graph Engineer Manager||Accenture||Atlanta, GA||Feb 14|
|Knowledge Graph Engineer Lead||Accenture||Philadelphia, PA||Feb 14|
|Data Science Consultant - Supply Chain||Accenture||Atlanta, GA||Feb 14|
|Knowledge Graph Engineer Lead||Accenture||Atlanta, GA||Feb 14|
|Knowledge Graph Engineer Manager||Accenture||Miami, FL||Feb 14|
|Software Engineer Team Lead (Java/Python)||Bazaarvoice||Austin, TX||Feb 14|
|Senior Data Engineer||Quest Groups LLC||Austin, TX||Feb 14|
|Thermal Analysis Engineer||Jacobs||Houston, TX||Feb 14|
|Data Scientist||Forcepoint||San Diego, CA||Feb 14|
|Data Analytics Consultant||TalentBridge||Minneapolis, MN||Feb 14|
|Senior Data Analyst - Consumer Product Analytics||RentPath||Atlanta, GA||Feb 14|
|Senior Manager Credit Risk Modeling & Analysis||Smith Hanley Associates||Detroit, MI||Feb 14|
|Senior Data Scientist - 27675||Alion Science and Technology||San Diego, CA||Feb 14|
|Software Engineer||Revenue Analytics, Inc.||Atlanta, GA||Feb 14|
|Head of Data Science||Mughal & Associates||Miami, FL||Feb 14|
|Data Reporting Analyst||Brooksource||Phoenix, AZ||Feb 14|
|Business Intelligence Analyst||Brooksource||Atlanta, GA||Feb 14|
|Data Analyst||Brooksource||Atlanta, GA||Feb 14|
|Data Reporting Analyst||Brooksource||Phoenix, AZ||Feb 14|
|Business Intelligence Analyst||Calculated Hire||Atlanta, GA||Feb 14|
|Senior Computer Vision Engineer||Huawei Technologies||San Diego, CA||Feb 14|
|Data Scientist II||LexisNexis||Raleigh, NC||Feb 14|
|Enterprise Data Architect||MRE Consulting, Ltd.||Houston, TX||Feb 14|
|Data Engineer (Python)||Cypress HCM||Atlanta, GA||Feb 14|
|Data Scientist||Cypress HCM||San Diego, CA||Feb 14|
|Data Scientist||TracFone Wireless||Miami, FL||Feb 14|
|BI Analyst||Cottonwood Financial||Dallas, TX||Feb 14|
|Senior OIPA Configurator - (Remote)||Elan Partners||Dallas, TX||Feb 14|
|AI Architect||eHire||Atlanta, GA||Feb 14|
|Vice President of Institutional Research||Western Governors University||Salt Lake City, UT||Feb 14|
|Data Scientist||Tokio Marine HCC||Houston, TX||Feb 14|
|Sr. Principal Machine Learning Engineer (Contributor or Lead)||SBT Industries||Austin, TX||Feb 14|
|Scrum Master in Autonomous Driving Team||TomTom||Eindhoven, Netherlands||Feb 14|
|Software Development Engineer||Expedia, Inc.||Bellevue, WA||Feb 13|
|Applied Machine Learning Engineer||Roc Search||Austin, TX||Feb 13|
|Data Science Engineer||PsiKick||Ann Arbor, MI||Feb 13|
|Senior Data Scientist||SparkCognition||Austin, TX||Feb 13|
|Concept Development Engineer with an edge towards embedded systems||Axis Communications||Lund, Sweden||Feb 13|
|Concept Development Engineer with an edge towards embedded systems||Axis Communications||Lund, Sweden||Feb 13|
|(Senior) Data Scientist||Sandvik||Stockholm, Sweden||Feb 13|
|Software Engineer Java, Machine Learning, NLP||Cortical.io||Vienna, Austria||Feb 13|
|Data Scientist Identity||trivago N.V.||Düsseldorf, Germany||Feb 13|
|Software Engineer: Signal Processing / Machine Learning||Applied Research in Acoustics LLC||Washington, DC||Feb 12|
|Entwickler (m/w) in Radar im Bereich Autonomes Fahren in Stuttgart||Daimler AG||Stuttgart, Deutschland||Feb 12|
|Data Scientist||Koninklijke Bibliotheek||The Hague, Netherlands||Feb 12|
|Product Owner - Value Products||GeoPhy||New York, NY||Feb 12|
|Machine Learning Engineer||GeoPhy||New York, NY||Feb 12|
|Machine Learning - Software Engineer / Scientist||Brain Corporation||San Diego, CA||Feb 12|
|Data Specialist||Marathon TS||Atlanta, GA||Feb 12|
|Develop AI and ML models including DNN, CNN, RNN, LSTM and GRU||Arm||Manchester, UK||Feb 12|
|Principal Data Scientist||Accenture||Atlanta, GA||Feb 12|
|Principal Data Scientist||Accenture||Houston, TX||Feb 12|
|Principal Data Scientist||Accenture||Dallas, TX||Feb 12|
|Big Data Engineer (m/w/d)||Giesecke+Devrient Currency Technology GmbH||München, Deutschland||Feb 11|
|Data Scientist Getafe||AIRBUS||Getafe, Spain||Feb 11|
|Machine Learning & Computer Vision Engineer (f/m)||Virtual Retail GmbH||Siegen, Deutschland||Feb 11|
|Principal Consultant - Data and Analytics||AIM Consulting Group||Minneapolis, MN||Feb 11|
|Director, Product and Data Science||Code North America||Dallas, TX||Feb 11|
|Senior Software Engineer||Rasa||Berlin, Germany||Feb 10|
|Senior NLP / Machine Learning Engineer||Avanoo (San Francisco, CA)||allows remote||Feb 10|
|Software Development Engineer - Full Stack||Expedia, Inc.||Bellevue, WA||Feb 9|
|Junior Programmer||Bongfish GmbH||Graz, Österreich||Feb 9|
|Director - Data & Analytics||GE Aviation||Austin, TX||Feb 9|
|Senior Software Engineer - Machine Control Software||Nordson Corporation||Colchester, UK||Feb 9|
|Backend Developer (PHP, Websockets, Machine learning, Utrecht)||Rentman||Utrecht, Netherlands||Feb 9|
|(Junior) Scala Developer||Cognotekt GmbH||Köln, Deutschland||Feb 9|
|Data Scientist (m/w) für die Intralogistik||viastore SOFTWARE GmbH||Stuttgart, Deutschland||Feb 9|
|Machine Learning Engineer||Mobilyte Inc.||Dallas, TX||Feb 9|
|Data Scientist||DecisionLogic (dba for Clarilogic. Inc.)||San Diego, CA||Feb 9|
|Data Scientist||Esri||New Delhi, India||Feb 9|
|Senior Backend Generalist||Revl, Inc. (San Francisco, CA)||allows remote||Feb 9|
|Siri Machine Learning Engineer||Apple||Cupertino, CA||Feb 8|
|Machine Learning Scientist - News||Apple||Cupertino, CA||Feb 8|
|System Engineer Machine Learning (f/m/div)*||Infineon||Munich, Germany||Feb 8|
|Data Scientist||Dutch Analytics .||Den Haag, Netherlands||Feb 8|
|Java Software Engineer||Semantic Web Company||Wien, Austria||Feb 8|
|Machine Learning Engineer||GeoPhy||Delft, Netherlands||Feb 8|
|Big Data Engineer||Grid Dynamics||Cupertino, CA||Feb 8|
|Ingenieure (w/m/d) / Naturwissenschaftler (w/m/d)als Europäische Patentprüfer (w||Das Europäische Patentamt||Den Haag, Niederlande||Feb 8|
|Team Lead Artificial Intelligence (m/f/d) E-Commerce||Invia Travel Germany GmbH||Leipzig, Deutschland||Feb 8|
|Machine Learning Engineer||MSH Talent Solutions||Phoenix, AZ||Feb 8|
|Software Engineer, Natural Language Processing and Machine Learning||Apple Inc.||Cupertino, CA||Feb 8|
|Data Scientist||Expedia Group - Europe||London, UK||Feb 7|
|AI Developer||Intact||Montreal, QC, Canada||Feb 7|
|Data Scientist II||Expedia, Inc.||Austin, TX||Feb 7|
|Softwareentwickler dessen Herz für Voice Technologien schlägt||Lindenbaum GmbH||Karlsruhe, Deutschland||Feb 7|
|Manager of Mobile Development (Android)||Viacom||Warsaw, Poland||Feb 7|
|Machine Learning Engineer Energy Market (Closing Date 7th March 2019)||Fetch.AI||Cambridge, UK||Feb 7|
|Do you have what it takes to save the Pony? Take our Full Stack code challenge!||Trustpilot||København, Denmark||Feb 7|
|Bioinformatics Scientist||Planet Pharma||San Diego, CA||Feb 7|
|Lead Data Scientist||CenturyLink||Littleton, CO||Feb 7|
|Applied Machine Learning Engineer||Alegion||Austin, TX||Feb 7|
|Tech founder (London)||Antler||London, UK||Feb 6|
|Head of Product||CANDIS||Berlin, Germany||Feb 6|