|
| 1 | +# Azure Data Tables client library for Python |
| 2 | + |
| 3 | +Azure Data Tables is a NoSQL data storing service that can be accessed from anywhere in the world via authenticated calls using HTTP or HTTPS. |
| 4 | +Tables scale as needed to support the amount of data inserted, and allow for the storing of data with non-complex accessing. |
| 5 | +Tables scale as needed to support the amount of data inserted, and allow for the storing of data with non-complex accessing. |
| 6 | +The Azure Data Tables client can be used to access Azure Storage or Cosmos accounts. |
| 7 | + |
| 8 | +Common uses of Azure Data Tables include: |
| 9 | + |
| 10 | +* Storing structured data in the form of tables |
| 11 | +* Quickly querying data using a clustered index |
| 12 | + |
| 13 | +[Source code](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk) | [Package (PyPI)](https://pypi.org) | [API reference documentation](https://aka.ms/azsdk/python/tables/docs) | [Product documentation](https://docs.microsoft.com/azure/storage/) | [Samples](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk) |
| 14 | + |
| 15 | +## Getting started |
| 16 | + |
| 17 | +### Prerequisites |
| 18 | +* Python 2.7, or 3.5 or later is required to use this package. |
| 19 | +* You must have an [Azure subscription](https://azure.microsoft.com/free/) and an |
| 20 | +[Azure storage account](https://docs.microsoft.com/azure/storage/common/storage-account-overview) to use this package |
| 21 | + or you must have a [Azure Cosmos Account](https://docs.microsoft.com/en-us/azure/cosmos-db/account-overview). |
| 22 | + |
| 23 | +### Install the package |
| 24 | +Install the Azure Data Tables client library for Python with [pip](https://pypi.org/project/pip/): |
| 25 | + |
| 26 | +```bash |
| 27 | +pip install --pre azure-data-tables |
| 28 | +``` |
| 29 | + |
| 30 | +### Create a storage account |
| 31 | +If you wish to create a new cosmos storage account, you can use the [Azure Cosmos DB](https://docs.microsoft.com/en-us/azure/cosmos-db/create-cosmosdb-resources-portal) |
| 32 | +If you wish to create a new storage account, you can use the |
| 33 | +[Azure Portal](https://docs.microsoft.com/azure/storage/common/storage-quickstart-create-account?tabs=azure-portal), |
| 34 | +[Azure PowerShell](https://docs.microsoft.com/azure/storage/common/storage-quickstart-create-account?tabs=azure-powershell), |
| 35 | +or [Azure CLI](https://docs.microsoft.com/azure/storage/common/storage-quickstart-create-account?tabs=azure-cli): |
| 36 | + |
| 37 | +```bash |
| 38 | +# Create a new resource group to hold the storage account - |
| 39 | +# if using an existing resource group, skip this step |
| 40 | +az group create --name MyResourceGroup --location westus2 |
| 41 | + |
| 42 | +# Create the storage account |
| 43 | +az storage account create -n mystorageaccount -g MyResourceGroup |
| 44 | +``` |
| 45 | + |
| 46 | +### Create the client |
| 47 | +The Azure Data Tables client library for Python allows you to interact with two types of resources: the |
| 48 | +account and tables, and entities. Interaction with these resources starts with an instance of a [client](#clients). |
| 49 | +To create a client object, you will need the account's table service endpoint URL and a credential that allows |
| 50 | +you to access the account: |
| 51 | + |
| 52 | +```python |
| 53 | +from azure.data.tables import TableServiceClient |
| 54 | + |
| 55 | +service = TableServiceClient(account_url="https://<myaccount>.table.core.windows.net/", credential=credential) |
| 56 | +``` |
| 57 | + |
| 58 | +#### Looking up the account URL |
| 59 | +You can find the account's table service URL using the |
| 60 | +[Azure Portal](https://docs.microsoft.com/azure/storage/common/storage-account-overview#storage-account-endpoints), |
| 61 | +[Azure PowerShell](https://docs.microsoft.com/powershell/module/az.storage/get-azstorageaccount), |
| 62 | +or [Azure CLI](https://docs.microsoft.com/cli/azure/storage/account?view=azure-cli-latest#az-storage-account-show): |
| 63 | + |
| 64 | +```bash |
| 65 | +# Get the table service URL for the account |
| 66 | +az storage account show -n mystorageaccount -g MyResourceGroup --query "primaryEndpoints.table" |
| 67 | +``` |
| 68 | + |
| 69 | +#### Types of credentials |
| 70 | +The `credential` parameter may be provided in a number of different forms, depending on the type of |
| 71 | +[authorization](https://docs.microsoft.com/azure/storage/common/storage-auth) you wish to use: |
| 72 | +1. To use a [shared access signature (SAS) token](https://docs.microsoft.com/azure/storage/common/storage-sas-overview), |
| 73 | + provide the token as a string. If your account URL includes the SAS token, omit the credential parameter. |
| 74 | + You can generate a SAS token from the Azure Portal under "Shared access signature" or use one of the `generate_sas()` |
| 75 | + functions to create a sas token for the account or table: |
| 76 | + |
| 77 | +```python |
| 78 | + from datetime import datetime, timedelta |
| 79 | + from azure.data.tables import TableServiceClient, generate_account_sas, ResourceTypes, AccountSasPermissions |
| 80 | + |
| 81 | + sas_token = generate_account_sas( |
| 82 | + account_name="<account-name>", |
| 83 | + account_key="<account-access-key>", |
| 84 | + resource_types=ResourceTypes(service=True), |
| 85 | + permission=AccountSasPermissions(read=True), |
| 86 | + expiry=datetime.utcnow() + timedelta(hours=1) |
| 87 | + ) |
| 88 | + |
| 89 | + table_service_client = TableServiceClient(account_url="https://<my_account_name>.table.core.windows.net", credential=sas_token) |
| 90 | +``` |
| 91 | + |
| 92 | +2. To use an account [shared key](https://docs.microsoft.com/rest/api/storageservices/authenticate-with-shared-key/) |
| 93 | + (aka account key or access key), provide the key as a string. This can be found in the Azure Portal under the "Access Keys" |
| 94 | + section or by running the following Azure CLI command: |
| 95 | + |
| 96 | + ```az storage account keys list -g MyResourceGroup -n mystorageaccount``` |
| 97 | + |
| 98 | + Use the key as the credential parameter to authenticate the client: |
| 99 | + ```python |
| 100 | + from azure.data.tables import TableServiceClient |
| 101 | + service = TableServiceClient(account_url="https://<my_account_name>.table.core.windows.net", credential="<account_access_key>") |
| 102 | + ``` |
| 103 | + |
| 104 | +#### Creating the client from a connection string |
| 105 | +Depending on your use case and authorization method, you may prefer to initialize a client instance with a |
| 106 | +connection string instead of providing the account URL and credential separately. To do this, pass the |
| 107 | +connection string to the client's `from_connection_string` class method: |
| 108 | + |
| 109 | +```python |
| 110 | +from azure.data.tables import TableServiceClient |
| 111 | + |
| 112 | +connection_string = "DefaultEndpointsProtocol=https;AccountName=xxxx;AccountKey=xxxx;EndpointSuffix=core.windows.net" |
| 113 | +service = TableServiceClient.from_connection_string(conn_str=connection_string) |
| 114 | +``` |
| 115 | + |
| 116 | +The connection string to your account can be found in the Azure Portal under the "Access Keys" section or by running the following CLI command: |
| 117 | + |
| 118 | +```bash |
| 119 | +az storage account show-connection-string -g MyResourceGroup -n mystorageaccount |
| 120 | +``` |
| 121 | + |
| 122 | +## Key concepts |
| 123 | +The following components make up the Azure Data Tables Service: |
| 124 | +* The account |
| 125 | +* A table within the account, which contains a set of entities |
| 126 | +* An entity within a table, as a dictionary |
| 127 | + |
| 128 | +The Azure Data Tables client library for Python allows you to interact with each of these components through the |
| 129 | +use of a dedicated client object. |
| 130 | + |
| 131 | +### Clients |
| 132 | +Two different clients are provided to to interact with the various components of the Table Service: |
| 133 | +1. [TableServiceClient](https://aka.ms/azsdk/python/tables/docs) - |
| 134 | + this client represents interaction with the Azure account itself, and allows you to acquire preconfigured |
| 135 | + client instances to access the tables within. It provides operations to retrieve and configure the account |
| 136 | + properties as well as query, create, and delete tables within the account. To perform operations on a specific table, |
| 137 | + retrieve a client using the `get_table_client` method. |
| 138 | +2. [TableClient](https://aka.ms/azsdk/python/tables/docs) - |
| 139 | + this client represents interaction with a specific table (which need not exist yet). It provides operations to |
| 140 | + create, delete, or update a table and includes operations to query, get, and upsert entities |
| 141 | + within it. |
| 142 | + |
| 143 | +### Entities |
| 144 | +* **Create** - Adds an entity to the table. |
| 145 | +* **Delete** - Deletes an entity from the table. |
| 146 | +* **Update** - Updates an entities information by either merging or replacing the existing entity. |
| 147 | +* **Query** - Queries existing entities in a table based off of the QueryOptions (OData). |
| 148 | +* **Get** - Gets a specific entity from a table by partition and row key. |
| 149 | +* **Upsert** - Merges or replaces an entity in a table, or if the entity does not exist, inserts the entity. |
| 150 | + |
| 151 | +## Examples |
| 152 | + |
| 153 | +The following sections provide several code snippets covering some of the most common Table tasks, including: |
| 154 | + |
| 155 | +* [Creating a table](#creating-a-table "Creating a table") |
| 156 | +* [Creating entities](#creating-entities "Creating entities") |
| 157 | +* [Querying entities](#querying-entities "Querying entities") |
| 158 | + |
| 159 | + |
| 160 | +### Creating a table |
| 161 | +Create a table in your account |
| 162 | + |
| 163 | +```python |
| 164 | +from azure.data.tables import TableServiceClient |
| 165 | + |
| 166 | +table_service_client = TableServiceClient.from_connection_string(conn_str="<connection_string>") |
| 167 | +table_service_client.create_table(table_name="myTable") |
| 168 | +``` |
| 169 | + |
| 170 | +### Creating entities |
| 171 | +Create entities in the table |
| 172 | + |
| 173 | +```python |
| 174 | +from azure.data.tables import TableClient |
| 175 | + |
| 176 | +my_entity = {'PartitionKey':'part','RowKey':'row'} |
| 177 | + |
| 178 | +table_client = TableClient.from_connection_string(conn_str="<connection_string>", table_name="myTable") |
| 179 | +entity = table_client.create_entity(entity=my_entity) |
| 180 | +``` |
| 181 | + |
| 182 | +### Querying entities |
| 183 | +Querying entities in the table |
| 184 | + |
| 185 | +```python |
| 186 | +from azure.data.tables import TableClient |
| 187 | + |
| 188 | +my_filter = "text eq Marker" |
| 189 | + |
| 190 | +table_client = TableClient.from_connection_string(conn_str="<connection_string>", table_name="mytable") |
| 191 | +entity = table_client.query_entities(filter=my_filter) |
| 192 | +``` |
| 193 | + |
| 194 | +## Optional Configuration |
| 195 | + |
| 196 | +Optional keyword arguments can be passed in at the client and per-operation level. The azure-core [reference documentation](https://azuresdkdocs.blob.core.windows.net/$web/python/azure-core/latest/azure.core.html) describes available configurations for retries, logging, transport protocols, and more. |
| 197 | + |
| 198 | + |
| 199 | +### Retry Policy configuration |
| 200 | + |
| 201 | +Use the following keyword arguments when instantiating a client to configure the retry policy: |
| 202 | + |
| 203 | +* __retry_total__ (int): Total number of retries to allow. Takes precedence over other counts. |
| 204 | +Pass in `retry_total=0` if you do not want to retry on requests. Defaults to 10. |
| 205 | +* __retry_connect__ (int): How many connection-related errors to retry on. Defaults to 3. |
| 206 | +* __retry_read__ (int): How many times to retry on read errors. Defaults to 3. |
| 207 | +* __retry_status__ (int): How many times to retry on bad status codes. Defaults to 3. |
| 208 | +* __retry_to_secondary__ (bool): Whether the request should be retried to secondary, if able. |
| 209 | +This should only be enabled of RA-GRS accounts are used and potentially stale data can be handled. |
| 210 | +Defaults to `False`. |
| 211 | + |
| 212 | +### Other client / per-operation configuration |
| 213 | + |
| 214 | +Other optional configuration keyword arguments that can be specified on the client or per-operation. |
| 215 | + |
| 216 | +**Client keyword arguments:** |
| 217 | + |
| 218 | +* __connection_timeout__ (int): Optionally sets the connect and read timeout value, in seconds. |
| 219 | +* __transport__ (Any): User-provided transport to send the HTTP request. |
| 220 | + |
| 221 | +**Per-operation keyword arguments:** |
| 222 | + |
| 223 | +* __raw_response_hook__ (callable): The given callback uses the response returned from the service. |
| 224 | +* __raw_request_hook__ (callable): The given callback uses the request before being sent to service. |
| 225 | +* __client_request_id__ (str): Optional user specified identification of the request. |
| 226 | +* __user_agent__ (str): Appends the custom value to the user-agent header to be sent with the request. |
| 227 | +* __logging_enable__ (bool): Enables logging at the DEBUG level. Defaults to False. Can also be passed in at |
| 228 | +the client level to enable it for all requests. |
| 229 | +* __headers__ (dict): Pass in custom headers as key, value pairs. E.g. `headers={'CustomValue': value}` |
| 230 | + |
| 231 | + |
| 232 | +## Troubleshooting |
| 233 | +### General |
| 234 | +Azure Data Tables clients raise exceptions defined in [Azure Core](https://github.com/Azure/azure-sdk-for-python/blob/master/sdk/core/azure-core/README.md). |
| 235 | +All Table service operations will throw a `HttpResponseError` on failure with helpful [error codes](https://docs.microsoft.com/en-us/rest/api/storageservices/table-service-error-codes). |
| 236 | + |
| 237 | +### Logging |
| 238 | +This library uses the standard |
| 239 | +[logging](https://docs.python.org/3/library/logging.html) library for logging. |
| 240 | +Basic information about HTTP sessions (URLs, headers, etc.) is logged at INFO |
| 241 | +level. |
| 242 | + |
| 243 | +Detailed DEBUG level logging, including request/response bodies and unredacted |
| 244 | +headers, can be enabled on a client with the `logging_enable` argument: |
| 245 | +```python |
| 246 | +import sys |
| 247 | +import logging |
| 248 | +from azure.data.tables import TableServiceClient |
| 249 | + |
| 250 | +# Create a logger for the 'azure.data.tables' SDK |
| 251 | +logger = logging.getLogger('azure.data.tables') |
| 252 | +logger.setLevel(logging.DEBUG) |
| 253 | + |
| 254 | +# Configure a console output |
| 255 | +handler = logging.StreamHandler(stream=sys.stdout) |
| 256 | +logger.addHandler(handler) |
| 257 | + |
| 258 | +# This client will log detailed information about its HTTP sessions, at DEBUG level |
| 259 | +service_client = TableServiceClient.from_connection_string("your_connection_string", logging_enable=True) |
| 260 | +``` |
| 261 | + |
| 262 | +Similarly, `logging_enable` can enable detailed logging for a single operation, |
| 263 | +even when it isn't enabled for the client: |
| 264 | +```py |
| 265 | +service_client.get_service_stats(logging_enable=True) |
| 266 | +``` |
| 267 | + |
| 268 | +## Next steps |
| 269 | + |
| 270 | +Get started with our [Table samples](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk). |
| 271 | + |
| 272 | +Several Azure Data Tables Python SDK samples are available to you in the SDK's GitHub repository. These samples provide example code for additional scenarios commonly encountered while working with Tables: |
| 273 | + |
| 274 | +* [table_samples_authentication.py](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk) - Examples found in this article: |
| 275 | + * From a connection string |
| 276 | + * From a shared access key |
| 277 | + * From a shared access signature token |
| 278 | +* [table_samples_service.py](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk) - Examples found in this article: |
| 279 | + * Get and set service properties |
| 280 | + * List tables in a account |
| 281 | + * Create and delete a table from the service |
| 282 | + * Get the TableClient |
| 283 | +* [table_samples_client.py](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk) - Examples found in this article: |
| 284 | + * Client creation |
| 285 | + * Create a table |
| 286 | + * Create and Delete entities |
| 287 | + * Query entities |
| 288 | + * Update entities |
| 289 | + * Upsert entities |
| 290 | + |
| 291 | +### Additional documentation |
| 292 | +For more extensive documentation on Azure Data Tables, see the [Azure Data Tables documentation](https://docs.microsoft.com/azure/storage/tables/) on docs.microsoft.com. |
| 293 | + |
| 294 | +## Contributing |
| 295 | +This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com. |
| 296 | + |
| 297 | +When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA. |
| 298 | + |
| 299 | +This project has adopted the [Microsoft Open Source Code of Conduct ](https://opensource.microsoft.com/codeofconduct/). For more information see the [Code of Conduct FAQ ](https://opensource.microsoft.com/codeofconduct/faq/) or contact [[email protected]](mailto:[email protected]) with any additional questions or comments. |
0 commit comments