Skip to content
Amar Nath and Shashi KhoslaSchool of Information TechnologyIIT Delhi
Project

Data flow and algorithm management for geospatial data processing

Version control and recomputation standards for DAGs of datasets and algorithms that span distributed hosting and compute nodes, including GPU-scaled algorithms.

ProjectOngoingSustainability and social development

Building geospatial datasets and indicators can involve complex data flow pipelines that start with base datasets, on which algorithms including data-driven algorithms like ML models are applied, to produce new downstream datasets, on which further algorithms are applied, and so on. This results in a directed acyclic graph on which data and algorithm version control is critical to implement to manage the datasets. If a base dataset changes or an intermediate algorithm is updated, it should trigger a re-computation in impacted downstream paths. Similarly, some of these datasets are temporal in nature and need to be updated on a regular frequency. A thesis that builds out these data and algorithm standards, generalizes it to operate over the web so that the data flow and algorithm graph can span a distributed system of multiple data hosting and computation nodes, allows scaling to novel data flow and algorithm graphs built by AI agents, and implements scalable versions of many algorithms that can leverage GPUs, will be a very relevant contribution.

Aaditeshwar Seth

  • PhD
  • MS (Research)
  • Pre-doc / Research Fellow

Python, data science, data structures, passion for societal impact

Website