datapi-core documentation

Welcome to the documentation for dataPi, a Python package that allows you to implement a distributed datalakehouse head made of data pods.

About dataPi

dataPi (from data + API) allows you to define, deploy data pods, list them, and generate documentation. It enables application developers to specify their informational query needs in a simple YAML file and creates dataPods - container-based deployable units that contain a local engine to resolve queries.

Contents:

• Installation
  • Installation from source
  • Verifying the Installation
  • Next Steps
• How It Works
  • Core Concepts
  • Workflow
  • Supported Technologies
  • Query Sources
  • dataPod Types
  • Next Steps
• Getting Started with dataPi
  • Installation
  • Initializing a New Project
  • Configuration
  • Defining Resources
  • Basic Commands
  • Using the Python Client SDK
• Commands
  • Initialize a New Project
  • Deploy Resources
  • List Resources
  • Generate Documentation
  • Serve Documentation
• Data Access from Applications
  • REST API Endpoint
  • Python Client SDK
  • Error Handling
  • Security Considerations
  • Next Steps
• Planned Features
  • Query Operations
  • Metastore Support
  • Container Building and Deployment
  • Data Transformations
  • dataPod Dependencies
  • User Interface
  • Embedded Engines
  • Resource Generation
  • Data Formats

Installation

For installation instructions, see Installation.

How it Works

Learn about the core concepts and workflow of dataPi in the How It Works section.

Getting Started

Get up and running with dataPi by following our Getting Started with dataPi guide.

Commands

Explore the various commands available in dataPi in the Commands section.

Data Access from Applications

Learn how to access data from your applications using dataPi in the Data Access from Applications section.

Planned Features

Check out the upcoming features and improvements planned for dataPi in the Planned Features section.

Indices and tables

• Index

• Module Index

• Search Page