Naveen Somasundaram

At Uber, we use robust data processing systems such as Apache Flink and Apache Spark to power the streaming applications that helps us calculate up-to-date pricing, enhance driver dispatching, and fight fraud on our platform.

Urban computing, a discipline that connects engineering, analysis, and visualization of data to improve safety, road infrastructure, equitable access to transportation, energy consumption, and traffic congestion in cities, is integral to Uber’s vision for safer, smarter cities.

The ability to manage and access petabytes of data quickly is crucial to the scalable growth of an entire data ecosystem. Still, this combined need for scale and speed does not always naturally fit into existing batch and streaming system architectures. Developed in 2016 under the codename “Hoodie,” Hudi was built to address inefficiencies across ingest and ETL pipelines that required upsert and incremental consumption primitives in Uber’s Big Data ecosystem.