ML/Cloud based system that efficiently analyzes collected data to predict/prevent/troubleshoot system failures and performance issues in smart-devices
Multi-tenancy & medium-high data volume processing
Data collected from smart devices is accessed from cloud (AWS) storage and undergoes translation from device-specific schema, file formats, etc and transformations such as selection of relevant data and features before being applied to a ML model training subsystem; qualified models are then pushed to production environment for prediction/execution. Data handling employs scalable spark-based access. The entire processing workflow is kept in sync via pipelines defined in airflow.
The state of the entire data engineering (& ML models, training and execution) is available via Dashboard UI
mazowieckie / Warszawa
Twój zakres obowiązków to: Analiza i integracja danych z zewnętrznych źródeł Monitorowania zmian w modelu danych Tworzenie, rozwijanie oraz monitorowanie procesów ETL Przygotowywania dokumentacji technicznej i użytkowej procesów ETL Nasze wymagania: bardzo...Więcej
Help create the design/architecture of the new data platform Do development in an Agile framework following the principles of Agile • Take up the User Stories that are created and made available in the Product Backlog Participate in the requirement discussion with...Więcej