Planned Features

Planned Features#

dataPi is continuously evolving to meet the needs of its users. Here’s a list of features and improvements we’re planning to implement in future releases:

Query Operations#

  • Add more operators, including JOINS

Metastore Support#

  • Add support for other metastores like Unity and BigQuery metastore

Container Building and Deployment#

  • Add support for more container building services, such as local Docker

  • Expand support for container deployment infrastructure, including Kubernetes

Data Transformations#

  • Add support for local transformations using dbt (data build tool)

dataPod Dependencies#

  • Enable dataPods to depend on other dataPods, not just tables

User Interface#

  • Add a UI for viewing deployed dataPods and their exposed contracts

Embedded Engines#

  • Add support for other embedded engines like Polars and Fusion

Resource Generation#

  • Implement automatic generation of resources using embedded LLMs (Language Models)

Data Formats#

  • Add support for sending data in more formats (e.g., JSON, Arrow)

Contents