Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
To send data to local storage in Azure IoT Operations, you can configure a data flow endpoint. With this configuration, you can specify the endpoint, authentication, table, and other settings.
Prerequisites
- An instance of Azure IoT Operations deployed in a Kubernetes cluster. For more information, see Deploy Azure IoT Operations.
The Azure CLI version 2.62.0 or newer installed on your development machine. Use
az --versionto check your version andaz upgradeto update if necessary. For more information, see Install the Azure CLI.The Azure IoT Operations extension for the Azure CLI. Use the following command to add the extension or update it to the latest version:
az extension add --upgrade --name azure-iot-ops
Create a local storage data flow endpoint
Use the local storage option to send data to a locally available persistent volume. You can use it to upload data via Azure Container Storage enabled by Azure Arc edge volumes.
In the operations experience, select the Data flow endpoints tab.
Under Create new data flow endpoint, select Local Storage > New.
Enter the following settings for the endpoint:
Setting Description Name The name of the data flow endpoint. Persistent volume claim name The name of the PVC to use for local storage. Select Apply to provision the endpoint.
The PVC must be in the same namespace as DataflowEndpoint.
Supported serialization formats
The only supported serialization format is Parquet.
Use Azure Container Storage enabled by Azure Arc
You can use the local storage data flow endpoint together with Azure Container Storage enabled by Azure Arc to store data locally or send data to a cloud destination.
Important
You must install Azure Container Storage enabled by Azure Arc before you use it with a local storage data flow endpoint.
Local shared volume
To write to a local shared volume, first create a PVC according to the instructions in Local shared edge volumes.
When you configure your local storage data flow endpoint, input the PVC name under persistentVolumeClaimRef.
Cloud ingest
To write your data to the cloud, follow the instructions in Cloud ingest edge volumes configuration to create a PVC and attach a subvolume for the cloud destination that you want.
To configure cloud ingest, your cluster must have secure settings enabled. The cloud ingest feature relies on workload identity federation.
Important
Don't forget to create the subvolume after you create the PVC. Otherwise, the data flow fails to start and the logs show a "read-only file system" error.
When you configure your local storage data flow endpoint, input the PVC name under persistentVolumeClaimRef.
Finally, when you create the data flow, the data destination parameter must match the spec.path parameter that you created for your subvolume during configuration.