Use this API to populate an existing dataset with new data. This API allows you to insert or append rows of data into a pre-defined dataset, making it useful for updating data during your projects to keep ML models in Graphite Note with up to date datasets.
Make a POST request to /dataset-fill with the required parameters in the request body.
Include the user-code and dataset-code to identify the dataset and the user making the request.
Define the structure of the data by specifying the columns parameter, which includes details like column names, aliases, types, subtypes, and optional formats.
Provide the data to be inserted via the insert-data parameter.
If compressed is false, the data should be formatted as a JSON-escaped string.
If compressed is true, the data should be base64-encoded after being gzipped.
If you have large dataset, it is a good idea to call dataset-fill in batch sizes of 10.000 rows, for example. Working example is below.
Example of insert-data with JSON (when compressed: false):
Use the append parameter to indicate whether to append the data to the existing dataset (true) or truncate the dataset before inserting the new data (false).
use the compressed parameter to specify if the data is gzip compressed (true) or not (false).
Example Usage
Filling a Dataset with Base64-encoded Data
For example, making a POST request to the following URL with the provided JSON body would result in the response below:
The response will confirm the status of the operation, including the dataset code and the number of rows inserted.
Sample Python Code
This Python script reads data from a PostgresSQL, converts it to JSON, compresses it using gzip, and encodes it as a Base64 string, ready to be sent via API requests.
Then It sends to Graphite Note in batch size of 10.000 rows.