AI Data Engineer

Overview

Job Title: AI Data Engineer

Location: Remote in the U.S. (CST. hours required)
Job Type: Full-Time Hourly
 We are looking for an AI Data Engineer to join our client’s team and contribute to the development of AI-driven systems using large-scale data solutions.

Job Overview:

We are seeking a highly skilled and motivated AI Data Engineer with experience in Open AI, Python, PySpark, Langchain, large data sets, AWS, and expertise in SQL queries.

This role will involve working on the design, development, and maintenance of robust data pipelines, facilitating the integration of AI models, and processing large-scale datasets in cloud-based environments.

Key Responsibilities:

• Data Engineering: Design, develop, and manage data pipelines to handle and process large datasets with a focus on scalability and efficiency.
• Langchain Integration: Utilize Langchain to build and optimize AI workflows, automate processes, and enhance AI model capabilities with data-driven features.
• Cloud Data Infrastructure: Manage and scale data infrastructure on AWS, using services such as S3, EC2, Lambda, Redshift, and others.
• SQL & Database Expertise: Write complex SQL queries for data extraction, transformation, and analysis. Optimize database performance and ensure data integrity.
• Collaboration: Work closely with Data Scientists, AI Engineers, and other stakeholders to ensure data is properly structured for machine learning models and other analytics use cases.
• Data Quality & Governance: Monitor data quality, create processes for error handling, and ensure compliance with data privacy regulations and best practices.
• Performance Optimization: Continuously optimize and fine-tune data processing pipelines and database queries for better performance and scalability.
• Documentation: Maintain detailed documentation on the architecture, data processes, and code to ensure smooth collaboration and knowledge transfer.

Requirements:

• Experience:
  • 3+ years of experience as a Data Engineer, with a focus on AI or machine learning data pipelines.
  • Hands-on experience with Langchain for building and optimizing AI workflows and automation.
  • Strong experience with large-scale data management, including working with distributed systems and processing massive datasets.
  • Proficient in SQL and experience with relational databases (e.g., PostgreSQL, MySQL, SQL Server) and NoSQL databases.
  • Extensive experience with AWS
  •  
• Technical Skills:
  • Proficient in Python and PySpark
  • Strong understanding of database design, data modeling, and query optimization.
  • Familiarity with data warehousing concepts and tools.
  • Experience working with data pipelines and workflow orchestration tools like Apache Airflow, or similar.
  • Knowledge of AI and machine learning concepts, particularly around data preprocessing and feature engineering.
•  
• Education:
  • Bachelor’s or Master’s degree in Computer Science, Data Engineering, Artificial Intelligence, or a related field.
•  
• Soft Skills:
  • Strong problem-solving and analytical skills.
  • Excellent communication skills, with the ability to explain technical concepts to non-technical stakeholders.
  • Ability to work in a fast-paced environment and manage multiple tasks effectively.
  • Team-oriented with a collaborative attitude.

Benefits:

• Subsidized Health, dental, and vision insurance.

If you’re passionate about data engineering, AI, and cloud-based technologies, and you thrive in an innovative and dynamic environment, we’d love to hear from you! Apply today and help us shape the future of AI-driven solutions.