Data Engineer – Data Observability & Pipelines

Fetcherr experts in deep learning, e-commerce, and digitization, Fetcherr disrupts traditional systems with its cutting-edge AI technology. At its core is the Large Market Model (LMM), an adaptable AI engine that forecasts demand and market trends with precision, empowering real-time decision-making. Specializing initially in the airline industry, Fetcherr aims to revolutionize industries with dynamic AI-driven solutions.

We are looking for a highly experienced and hands-on Data Engineer to lead the development of our internal data observability and orchestration ecosystem. This role combines strong technical execution with cross-organizational collaboration, playing a critical role in ensuring the reliability, traceability, and visibility of our data pipelines and assets.

You’ll design and own systems that detect issues before they impact production, visualize key data quality metrics, and provide internal stakeholders with intuitive access to data lineage, validation errors, and pipeline performance.

Responsibilities:

Pipeline Orchestration & Monitoring

• Design, build, and maintain scalable Dagster data pipelines.
• Integrate data validation checkpoints into a dedicated pipeline and connect alerts to Google Cloud Monitoring and incident.io
• Track and visualize data arrival status, pipeline execution results, and anomalies.

Data Observability & UI

• Build and maintain an internal UI dashboard to display:
• Data validation errors
• Pipeline execution status
• Time series trends (e.g., volume anomalies)
• Dataset arrival status
• Various prometheus KPIs
• Create visual and interactive tools to support investigation and resolution of data issues.

Data Management & Collaboration

• Implement data lineage, data cataloging, and metadata management across datasets.
• Enable controlled editability and auditability of BigQuery tables through intuitive tools.
• Collaborate across R&D, Product, and Data Science teams to align on standards and practices.