Skip to content

Commit 97c43d1

Browse files
xiafu-msfttasherif-msftazure-sdkchidozieononiwuaviramha
authored
Feature/storage stg74 (#14175)
* [Storage][Generate]Generate Blob, Share, Datalake code * regenerate code * reverted undelete container changes (#13377) * [ADLS]change api version * Added Last Access Time Feature (#13433) * converted BlobItemInternal to BlobProperties and added the attribute in model's BlobProperties * removed none default * added unit test for get properties with lat * more unit tests * fixed failing test * added docstrings and addeed extra test * Update sdk/storage/azure-storage-blob/azure/storage/blob/_models.py Co-authored-by: Xiaoxi Fu <[email protected]> Co-authored-by: Xiaoxi Fu <[email protected]> * [Swagger]regenerate swagger for file-share * Added SMB Multichannel protocol (#13795) * added smb multichannel feature * added protocol properties to desarialize * added two more classes * better docstrings * added test and everything is working * added async * passing tests * fixed var names * Merge master again (#13967) * Sync eng/common directory with azure-sdk-tools repository for Tools PR 916 (#13374) * Update Language Settings File (#13389) * [Storage]Revert equal sign in async (#13501) * Sync eng/common directory with azure-sdk-tools repository for Tools PR 930 (#13384) * fix authorization header on asyncio requests containing url-encoded chars (#13346) * fix authorization header on asyncio requests containing url-encoded-able characters (=, ! etc) * edit all authentication files and add a test Co-authored-by: xiafu <[email protected]> * If match headers (#13315) * verifying match conditions are correct, adding tests for other conditions * changes to verify etag is working, think it's being ignored by service * another change to test file, passing the correct etag to the service, it returns a 204 so no error is produced * renaming vars * testing for if-match conditions to make sure they are properly parsed and sent to service * reverting header back * testing update to reflect new response in create_entity * [text analytics] link opinion mining in the readme (#13493) * modify bing id docstring to mention Bing Entity Search more (#13509) * Tableitem metadata (#13445) * started collecting metadata, issue with asynciterator not being iterable * simplifying code, improving testing * issue with one recording, reran and seems good to go * fixing doc strings * fixed two tests, skipped another one. AsyncPageIterator is missing something to make it an iterator * new recording and skipping one test until can figure out AsyncPageIterator issue * adding asserts * add test for date and table_name, api_version is not returned unless full_metadata is specified * [EventGrid] Receive Functions (#13428) * initial changes * comments * str * static * from_json * Update sdk/eventgrid/azure-eventgrid/azure/eventgrid/_helpers.py * Fix storage file datalake readme and samples issues (#12512) * Fix storage file datalake readme issues * Update README.md * Make doc-owner recommended revisions to eventhub readme and samples. (#13518) Move auth samples into samples and link from readme rather than line. Add snippets to the top of samples lacking any verbiage. Whitespace improvements in client auth sample Explicit samples for connstr auth to link from new terse readme auth section. * [ServiceBus] Clean up README prior to P6 with doc-owner recommendations. (#13511) * Remove long-form samples from readme authentication section and move them into formal samples with URIs. (This by recommendation of docs folks, and supported via UX interview with user stating that the README was getting long in the teeth with this section being less critical) * Live pipeline issues (#13526) * changes to the tests that reflects the new serialization of EntityProperties, failing on acl payloads and sas signed identifiers * re-generated code, fixed a couple test methods, solved the media type issue for AccessPolicies, re-recorded tests because of changes in EntityProperty * updating a recording that slipped through * 404 python erroring sanitize_setup. should not be (#13532) * Sync eng/common directory with azure-sdk-tools repository for Tools PR 946 (#13533) * update release date for Sep (#13371) * update release date for Sep * fix typo * update release date for Sep (#13370) * update changelog (#13369) * [text analytics] add --pre suffix to pip install (#13523) * Change prerelease versioning (#13500) - Use a instead of dev - Fix dev to alpha for regression test - Clean-up some devops feed publishing steps * add test for opinion in diff sentence (#13524) * don't want to exclude mgmt. auto-increments are fine here (#13549) * [keyvault] fix include_pending param and 2016-10-01 compatibility (#13161) * Add redirect_uri argument to InteractiveBrowserCredential (#13480) * Sync eng/common directory with azure-sdk-tools for PR 955 (#13553) * Increment package version after release of azure_storage_file_datalake (#13111) * Increment package version after release of azure_storage_file_share (#13106) * Clean up "?" if there is no query in request URL (#13530) * cleaned up the ? in source urls * [devtool]turn primary_endpoint and key from unicode into str Co-authored-by: xiafu <[email protected]> * Update readme samples (#13486) * updated each sample to use env variables, more informative print statements * updating readme with more accurate links, still missing a few * grammar fixes * added a readme, and async versions for auth and create/delete operations * added more files (copies for async), and a README to the samples portion * basic commit, merging others into one branch * put async into another folder * more README updates to align with .NET * initial draft of README and samples * initial comments from cala and kate * caught a text analytics reference, thanks kate * caught a text analytics reference, thanks kate * reverting version * fixing two broken links * recording for test_list_tables that got left out * fixing broken links * another attempt at broken links * changing parentheses to a bracket per kristas recommendation * commenting out one test with weird behavior * found an actual broken link * lint fixes * update to readme and samples per cala, issy, and kates recs. added examples to sphinx docs that were not there. * added a quote around odata filter, thanks Chris for the catch * updating a few more broken links * adding an aka.ms link for pypi * two more broken links * reverting a change back * fixing missing END statements, removing link to nowhere, correcting logger * Sync eng/common directory with azure-sdk-tools repository for Tools PR 926 (#13368) * [keyvault] add scope enum (#13516) * delete connection_string in recorded tests (#13557) * Sync eng/common directory with azure-sdk-tools repository for Tools PR 823 (#13394) * Add sample docstrings to eg samples (#13572) * docstrings * Apply suggestions from code review * few more docstrings * decode * add licesnse * [EventGrid] README.md updates (#13517) * Fix a link which did not have a target * Use "Event Grid" instead of "Eventgrid" or "EventGrid" in prose * Cloud Event Abstraction (#13542) * initial commit * oops * some changes * lint * test fix * sas key * some more cahgnes * test fix * 2.7 compat * comments * extensions * changes * Update sdk/eventgrid/azure-eventgrid/azure/eventgrid/_models.py * remove test * Increment version for storage releases (#13096) * Increment package version after release of azure_storage_blob * Fix a docstring problem (#13579) * SharedTokenCacheCredential uses MSAL when given an AuthenticationRecord (#13490) * [EventHub] add SAS token auth capabilities to EventHub (#13354) * Add SAS token support to EventHub for connection string via 'sharedaccesssignature' * Adds changelog/docs/samples/tests, for now, utilize the old-style of test structure for sync vs async instead of preparer until import issue is resolved. * [Evengrid] Regenrate code gen (#13584) * Regenrate swagger * fix import * regen * Update Language Settings file (#13583) * Added blob exists method (#13221) * added feature and unit tests * fixed failing test issue * added async method and more unit tests * ffixed passed parameters * fixed python 27 issue with kwargs * reset commit * Update _blob_client_async.py removed unused import, fixed linter * Update sdk/storage/azure-storage-blob/azure/storage/blob/aio/_blob_client_async.py Co-authored-by: Xiaoxi Fu <[email protected]> * Update sdk/storage/azure-storage-blob/azure/storage/blob/_blob_client.py Co-authored-by: Xiaoxi Fu <[email protected]> * fixed failing tests * fixed linting/import order Co-authored-by: Xiaoxi Fu <[email protected]> * [keyvault] administration package readme (#13489) * add release date to changelog (#13590) * Sync eng/common directory with azure-sdk-tools repository (#13589) * VisualStudioCodeCredential raises CredentialUnavailableError when configured for ADFS (#13556) * Implement full vault backup and restore (#13525) * Identity release notes (#13585) * [DataLake][Rename]Rename with Sas (#12057) * [DataLake][Rename]Rename with Sas * small fix * recordings * fix pylint * fix pylint * fix pylint * Update sdk/storage/azure-storage-file-datalake/azure/storage/filedatalake/aio/_data_lake_directory_client_async.py * [Schema Registry + Avro Serializer] 1.0.0b1 (#13124) * init commit * avro serializer structure * adding avro serializer * tweak api version and fix a typo * test template * avro serializer sync draft * major azure sr client work done * add sample docstring for sr * avro serializer async impl * close the writer * update avro se/de impl * update avro serializer impl * fix apireview reported error in sr * srav namespace, setup update * doc update * update doc and api * impl, doc update * partial update according to laruent's feedback * be consistent with eh extension structure * more update code according to feedback * update credential config * rename package name to azure-schemaregistry-avroserializer * fix pylint * try ci fix * fix test for py27 as avro only accept unicode * first round of review feedback * remove temp ci experiment * init add conftest.py to pass py2.7 test * laurent feedback update * remove dictmixin for b1, update comment in sample * update api in avroserializer and update test and readme * update test, docs and links * add share requirement * update avro dependency * pr feedback and livetest update * Win py27 issue (#13595) * change to the processor to check for BOM at the beginning * changes to serialize and deserialize to be consistent with py2/3 strings/unicode types. new recordings for all that, changes to tests for everything, and a bug in sas file * forgot to add a unicode explicitly, re-recorded some tests because of that * Sync eng/common directory with azure-sdk-tools repository for Tools PR 969 (#13591) * [ServiceBus] Expose internal amqp message properties via AMQPMessage wrapper object on Message (#13564) * Expose internal amqp message properties via AMQPMessage wrapper object on Message. Add test, changelog notes and docstring. (Note: Cannot rename old message as uamqp relies on the internal property name. Should likely be adapted.) Co-authored-by: Adam Ling (MSFT) <[email protected]> * Update doc link in README.md (#13618) * Bump template version (#13580) * Increment package version after release of azure_appconfiguration (#13620) * Increment package version after release of azure_search_documents (#13622) * Increment package version after release of azure_core (#13621) * Fix data nspkg (#13623) * Add data nspkg to CI (#13626) * Update EH and SB code owner (#13633) * Rename ServiceBusManagementClient to ServiceBusAdministrationClient (#13597) * Rename ServiceBusManagementClient to ServiceBusAdministrationClient * Increment package version after release of azure_keyvault_certificates (#13630) * Add administration package to Key Vault pipeline (#13631) * fixed the long description, addition to changelog (#13637) * fixed the long description, addition to changelog * addresssing izzy's comments * [EventGrid] Fix lint errors (#13640) * Fix lint errors * comments * * Remove `is_anonymous_accessible` from management entities. (#13628) * Remove `support_ordering` from `create_queue` and `QueueProperties` * Remove `enable_subscription_partitioning` from `create_topic` and `TopicProperties` * update tests/changelog * Eventgrid Prepare for Release (#13643) * Release preparation * shared * codeowners * [Schema Registry] Fix docstring and docs (#13625) * fix docstring and docs * update codeowner and ci config * update init in serializer * update readme * update sr dependecy in avro serializer * update module __init__.py * revert dependcy to see if it helps doc generetion * [ServiceBus] Replace get_*_deadletter_receiver functions with a sub_queue parameter (#13552) * Remove get_*_deadletter_receiver functions and add a sub_queue parameter to get_*_receiver functions taking enum SubQueue to specify the target subqueue. Adjusts tests, docs accordingly. Co-authored-by: Adam Ling (MSFT) <[email protected]> * [Storage]Fix a permission bug and add enable test for list blob with metadata (#13260) * Fix permission bug * test list blobs with metadata * add snapshot to get_blob_properties() response * fix test * [ChangeFeed]Unify cursor and add live mode (#13243) * [ChangeFeed]Unify cursor and add live mode * make the token into a str * address comments * 1. add a while True for sample 2. make the list of shards in cursor to a dict in internal code 3. test list 3-shard events in multiple times generate same results as list all events at once 4. Java is using sequential list, so it could give 1 shard cursor even there are 3 shards, the test makes sure python is working with 1 shard cursor. * make end_time/start_time and continuation_token mutual exclusive * update dependency version * make all '' to "" in cursor * fix pylint and update changelog * fix blob pylint * added playback mode only marker (#13636) * Update changelog (#13641) * added changelogs * added prs/issues in the changelogs * Update CHANGELOG.md (#13657) * - Rename `entity_availability_status` to `availability_status` (#13629) - Make it unsettable in create_* mgmt operations. (as it is readonly) - adjust tests, docs etc. * init resourcemover ci (#13666) Co-authored-by: xichen <[email protected]> * Sdk automation/track2 azure mgmt keyvault (#13662) * Generated from c273efbfeb4c2c2e0729579114947c91ab747daa add tag * version and test Co-authored-by: SDK Automation <[email protected]> * Increment package version after release of azure_keyvault_administration (#13651) * Increment package version after release of azure_identity (#13652) * [SchemaRegistry] small fix in setup.py (#13677) * [SchemaRegistry] Pin avro serializer dependency version (#13649) * Increment package version after release of azure_data_tables (#13642) * Increment version for eventhub releases (#13644) * Increment package version after release of azure_eventhub * Increment package version after release of azure_eventhub_checkpointstoreblob * Increment package version after release of azure_eventhub_checkpointstoreblob_aio * Add parameters to function (#13653) * [ServiceBus] Consistency review changes as detailed in issue #12415. (#13160) * Consistency review changes as detailed in issue #12415. * significant amount of renames, parameter removal, mgmt shim class building, and a few added capabilities in terms of renew_lock retval and receive_deferred param acceptance. * Update mgmt test recordings Co-authored-by: Adam Ling (MSFT) <[email protected]> * [SchemaRegistry] Re-enable links check (#13689) * Release sdk resourcemover (#13665) * Generated from b7867a975ec9c797332d735ed8796474322c6621 fix schemas parameter * init ci and version Co-authored-by: SDK Automation <[email protected]> Co-authored-by: xichen <[email protected]> * [ServiceBus] Support SAS token-via-connection-string auth, and remove ServiceBusSharedKeyCredential export (#13627) - Remove public documentation and exports of ServiceBusSharedKeyCredential until we chose to release it across all languages. - Support for Sas Token connection strings (tests, etc) - Add safety net for if signature and key are both provided in connstr (inspired by .nets approach) Co-authored-by: Rakshith Bhyravabhotla <[email protected]> * [text analytics] default to v3.1-preview.2, have it listed under enum V3_1_PREVIEW (#13708) * Sync eng/common directory with azure-sdk-tools repository for Tools PR 982 (#13701) * Remove locale from docs links (#13672) * [ServiceBus] Set 7.0.0b6 release date in changelog (#13715) * [ServiceBus] Sample Fix (#13719) * fix samples * revert duration * Increment package version after release of azure_schemaregistry_avroserializer (#13682) * Increment package version after release of azure_schemaregistry (#13679) * Revert the changes of relative links (#13681) * KeyVaultBackupClient tests (#13709) * Release sdk automanage (#13693) * add automanage ci * auto generated sdk * add init Co-authored-by: xichen <[email protected]> * Replace UTC_Now() workaround with MSRest.UTC (#13498) * use msrest.serialization utc instead of custom implementation * update reference for utc Co-authored-by: Andy Gee <[email protected]> * Increment package version after release of azure_servicebus (#13720) * Sync eng/common directory with azure-sdk-tools repository for Tools PR 965 (#13578) * Test preparer region config loader (and DeleteAfter fixes) (#12924) * Initial implementation of region-loading-from-config, and opting in for SB and EH to support canary region specification. * Truncate DeleteAfter at milliseconds to hopefully allow it to get picked up by engsys. Add get_region_override to __init__ exports. * Provide better validation semantics for the get_region_override function. (empty/null regions is invalid) * Sync eng/common directory with azure-sdk-tools for PR 973 (#13645) * Sync eng/common directory with azure-sdk-tools repository for Tools PR 973 * Update update-docs-metadata.ps1 * Update update-docs-metadata.ps1 Co-authored-by: Sima Zhu <[email protected]> * Update Tables Async Samples Refs (#13764) * remove dependency install from azure-sdk-tools * update tables samples w/ appropriate relative URLs * undo accidental commit * admonition in table service client not indented properly * Internal code for performing Key Vault crypto operations locally (#12490) * Abstract auth to the dev feeds. additionally, add pip auth (#13522) * abstract auth to the dev feeds. additionally, add pip auth * rename to yml-style filename formatting * [EventHubs] Make __init__.py compatible with pkgutil-style namespace (#13210) * Make __init__.py compatible with pkgutil-style namespace Fixes #13187 * fix pylint and add license info Co-authored-by: Yunhao Ling <[email protected]> * Raise msal-extensions dependency to ~=0.3.0 (#13635) * add redacted_text to samples (#13521) * adds support for enums by converting to string before sending on the … (#13726) * adds support for enums by converting to string before sending on the wire * forgot about considerations for python2 strings/unicode stuff * Allow skip publish DocMS or Github IO for each artifact (#13754) * Update codeowners file for Azure Template (#13485) * Sync eng/common directory with azure-sdk-tools repository for Tools PR 999 (#13791) * Sync eng/common directory with azure-sdk-tools repository for Tools PR 1000 (#13792) * regenerated with new autorest version (#13814) * removed try/except wrapper on upsert method, added _process_table_error instead of create call (#13815) fixes #13678 * Sync eng/common directory with azure-sdk-tools repository for Tools PR 974 (#13650) * [EventHubs] Update extensions.__ini__.py to the correct namespace module format (#13773) * Sync eng/common directory with azure-sdk-tools repository for Tools PR 895 (#13648) * unskip aad tests (#13818) * [text analytics] don't take doc # in json pointer to account for now bc of service bug (#13820) * Increment package version after release of azure_eventgrid (#13646) * [T2-GA] Appconfiguration (#13784) * generate appconfiguration track2 ga version * fix changelog * fix version * fix changelog * generate keyvault track2 ga version (#13786) * generate monitor track2 ga version (#13804) * generate eventhub track2 ga version (#13805) * generate network track2 ga version (#13810) * generate compute track2 ga version (#13830) * [T2-GA] Keyvault (#13785) * generate keyvault track2 ga version * fix changelog * Update CHANGELOG.md Co-authored-by: changlong-liu <[email protected]> * CryptographyClient can decrypt and sign locally (#13772) * update release date (#13841) * add link to versioning story in samples (#13842) * GeneralNameReplacer correctly handles bytes bodies (#13710) * [ServiceBus] Update relative paths in readme/migration guide (#13417) * update readme paths * update for urls Co-authored-by: Andy Gee <[email protected]> * Replaced relative link with absolute links and remove locale (#13846) Replaced relative link with absolute links and remove locale * Enable the link check on aggregate-report (#13859) * use azure-mgmt-core 1.2.0 (#13860) * [T2-GA] Resource (#13833) * ci.yml (#13862) * remove azure-common import from azure-devtools resource_testcase (#13881) * move import from azure-common to within the track1 call (#13880) * Synapse regenerated on 9/1 with autorest 5.2 preview (#13496) * Synapse regenerated on 9/1 with autorest 5.2 preview * 0.3.0 * ChangeLog * Update with latest autorest + latest Swagger 9/16 * use autorest.python 5.3.0 (#13835) * use autorest.python 5.3.0 * codeowner * 20200918 streamanalytics (#13861) * generate * add init.py * Small set of non-blocking changes from b6. (#13690) - adds EOF whitespace - renames _id to _session_id - Adjusts docstring type Closes #13686 * Add placeholder yml file for pipeline generation * Bump Storage-Blob Requires to Range (#13825) * bump dependency versions * update upper bound appropriately * revert unecessary changes * updated formatting to eliminate trailing comma. these setup.py don't do that * bump shared_requirements * [ServiceBus] mode (ReceiveMode) parameter needs better exception behavior (#13531) * mode parameter needs better exception behavior, as if it is mis-typed now it will return an AttributeError further down the stack without useful guidance (and only does so when creating the handler, as well). Will now return a TypeError at initialization. * Add note of AttributeError->TypeError behavior for receive_mode misalignment to changelog. * [SchemaRegistry] Samples for EH integration (#13884) * add sample for EH integration * add samples to readme and tweak the code * add descriptions * mention SR and serializer in EH * small tweak * Add communication service mapping * Update docs to reflect Track 2 Python SDK status (#13813) * Update python mgmt libraries message * Update and rename mgmt_preview_quickstart.rst to mgmt_quickstart.rst * Update python_mgmt_migration_guide.rst * Update index.rst * Update README.md * Update README.md * Update README.md * KeyVaultPreparer passes required SkuFamily argument (#13845) * Add code owners for Azure Communication Services (#13946) * Resolve Failing SchemaRegistry Regressions (#13817) * make the wheel retrieval a little bit more forgiving * add 1.0.0b1 to the omission * update version exclusion * add deprecate note to v1 of form recognizer (#13945) * add deprecate note to v1 of form recognizer * update language, add back to ci.yml * Additional Fixes from GA-ed Management Packages (#13914) * add adal to dev_reqs for storage package * add msrestazure to the dev_reqs for azure-common * azure-loganalytics and azure-applicationinsights are both still track1. have to add msrestazure to the dev_reqs as azure-common requires it * remove import of azure-common from the tests * bump the version for azure-mgmt-core. * crypto (#13950) * Added partition key param for querying change feed (#13857) * initia; changes for partitionkey for query changefeed * Added test * updated changelog * moved partition_key to kwargs * Sync eng/common directory with azure-sdk-tools repository for Tools PR 1022 (#13885) * Update testing (#13821) * changes for test_table.py * fixed up testing, noting which tests do not pass and the reasoning * small additions to testing * updated unicode test for storage * final update * updates that fix user_agent tests * test CI returns a longer user agent so flipping the order for this test should solve the issue * addresses anna's comments * re-recorded a test with a recording error * removed list comprehension for python3.5 compatability * fixing a testing bug * track2_azure-mgmt-baremetalinfrastructure for CI run normally (#13963) * ci.yml for track2_azure-mgmt-baremetalinfrastructure to make CI run normally * Removed unnecessary includes. Co-authored-by: Mitch Denny <[email protected]> Co-authored-by: Azure SDK Bot <[email protected]> Co-authored-by: Chidozie Ononiwu <[email protected]> Co-authored-by: Aviram Hassan <[email protected]> Co-authored-by: Sean Kane <[email protected]> Co-authored-by: iscai-msft <[email protected]> Co-authored-by: Rakshith Bhyravabhotla <[email protected]> Co-authored-by: Tong Xu (MSFT) <[email protected]> Co-authored-by: KieranBrantnerMagee <[email protected]> Co-authored-by: Scott Beddall <[email protected]> Co-authored-by: Xiang Yan <[email protected]> Co-authored-by: Wes Haggard <[email protected]> Co-authored-by: Charles Lowell <[email protected]> Co-authored-by: tasherif-msft <[email protected]> Co-authored-by: Matt Ellis <[email protected]> Co-authored-by: Yijun Xie <[email protected]> Co-authored-by: Adam Ling (MSFT) <[email protected]> Co-authored-by: Sima Zhu <[email protected]> Co-authored-by: Laurent Mazuel <[email protected]> Co-authored-by: xichen <[email protected]> Co-authored-by: xichen <[email protected]> Co-authored-by: changlong-liu <[email protected]> Co-authored-by: SDK Automation <[email protected]> Co-authored-by: Rakshith Bhyravabhotla <[email protected]> Co-authored-by: Andy Gee <[email protected]> Co-authored-by: Andy Gee <[email protected]> Co-authored-by: Piotr Jachowicz <[email protected]> Co-authored-by: Kaihui (Kerwin) Sun <[email protected]> Co-authored-by: Wes Haggard <[email protected]> Co-authored-by: nickzhums <[email protected]> Co-authored-by: turalf <[email protected]> Co-authored-by: Krista Pratico <[email protected]> Co-authored-by: Srinath Narayanan <[email protected]> Co-authored-by: msyyc <[email protected]> Co-authored-by: Mitch Denny <[email protected]> * Recursive acl (#13476) * [Storage][Datalake]Added supoort for recursive acl operations * [DataLake]Recursive ACL * re-record * re-record * rename progress_callback to progress_hook * re-record Co-authored-by: zezha-msft <[email protected]> * [Storage]API Review Comments (#14019) * [Storage]API Review Comments * move new_name for undelete_container to kwargs * [Storage][Blob][QuickQuery]Arrow Format (#13750) * [Storage][Blob][DataLake]Quick Query Arrow Format * fix pylint * fix pylint * fix pylint * fix pylint * Set expiry (#12642) * [DataLake][SetExpiry]Set Expiry of DataLake File * address comments * use datalake set_expiry operation * add serialize rfc1123 and fix pylint * fix pylint * remove return type * Added get file range with the prevsharesnapshot parameter (#13507) * added feature * fixed var name * added unit test that is still being worked on * added unit test for get file range with snapshot * added recording of the unit test * added async unit test recording * [Swagger][FileShare]Regenerate for Clear Range Change * added a deserialize method * recoreded new tests and added deserialize * rerecorded * recorded some more * changed tests for sync * recorded async tests * added more docstrings * linter * added additional api * linter * unused import linter Co-authored-by: xiafu <[email protected]> Co-authored-by: Xiaoxi Fu <[email protected]> * [Datalake][Exception]Throw DataLakeAclChangeFailedError (#14129) * [Datalake][Exception]Throw DataLakeAclChangeFailedError * fix pylint * fix pylint * Share Lease Feature (#13567) * added needed parameters for shares * added async methods * added more methods for interacting with the API * fixed small mistake with elif * added tests and access conditions * added more tests for leases * fixed tests * async changes * added await * corrected import * fixed async imports and wrote all tests * linting * share renaming * added file lease sample * added sample for share * fixed samples * added docs * removed checks * lease change * lease change * added correct lease durations * removed spacing * version correction * fixed snapshot * added snapshot tests * added snapshot tests * changed version * test * test * more docstrings * fixed docstrings * more docstring changes * removed etag * added exta check on test * fixed tests * added more tests for file * added tests for list shares * unused import * changed method signitures * fixed kwargs * linter Co-authored-by: Xiaoxi Fu <[email protected]> * removed changelog feature * [DelegationSas]support directory sas & add feature for delegation sas (#14206) * [DelegationSas]support directory sas & add feature for delegation sas * add doc string Co-authored-by: tasherif-msft <[email protected]> Co-authored-by: Azure SDK Bot <[email protected]> Co-authored-by: Chidozie Ononiwu <[email protected]> Co-authored-by: Aviram Hassan <[email protected]> Co-authored-by: Sean Kane <[email protected]> Co-authored-by: iscai-msft <[email protected]> Co-authored-by: Rakshith Bhyravabhotla <[email protected]> Co-authored-by: Tong Xu (MSFT) <[email protected]> Co-authored-by: KieranBrantnerMagee <[email protected]> Co-authored-by: Scott Beddall <[email protected]> Co-authored-by: Xiang Yan <[email protected]> Co-authored-by: Wes Haggard <[email protected]> Co-authored-by: Charles Lowell <[email protected]> Co-authored-by: Matt Ellis <[email protected]> Co-authored-by: Yijun Xie <[email protected]> Co-authored-by: Adam Ling (MSFT) <[email protected]> Co-authored-by: Sima Zhu <[email protected]> Co-authored-by: Laurent Mazuel <[email protected]> Co-authored-by: xichen <[email protected]> Co-authored-by: xichen <[email protected]> Co-authored-by: changlong-liu <[email protected]> Co-authored-by: SDK Automation <[email protected]> Co-authored-by: Rakshith Bhyravabhotla <[email protected]> Co-authored-by: Andy Gee <[email protected]> Co-authored-by: Andy Gee <[email protected]> Co-authored-by: Piotr Jachowicz <[email protected]> Co-authored-by: Kaihui (Kerwin) Sun <[email protected]> Co-authored-by: Wes Haggard <[email protected]> Co-authored-by: nickzhums <[email protected]> Co-authored-by: turalf <[email protected]> Co-authored-by: Krista Pratico <[email protected]> Co-authored-by: Srinath Narayanan <[email protected]> Co-authored-by: msyyc <[email protected]> Co-authored-by: Mitch Denny <[email protected]> Co-authored-by: zezha-msft <[email protected]>
1 parent 5f61290 commit 97c43d1

File tree

225 files changed

+77892
-2735
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

225 files changed

+77892
-2735
lines changed

eng/ci_tools.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
# requirements leveraged by ci tools
2+
cryptography==3.1
23
setuptools==44.1.0; python_version == '2.7'
34
setuptools==46.4.0; python_version >= '3.5'
45
virtualenv==20.0.23

sdk/storage/azure-storage-blob/azure/storage/blob/__init__.py

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -54,6 +54,8 @@
5454
BlobQueryError,
5555
DelimitedJsonDialect,
5656
DelimitedTextDialect,
57+
ArrowDialect,
58+
ArrowType,
5759
ObjectReplicationPolicy,
5860
ObjectReplicationRule
5961
)
@@ -219,6 +221,8 @@ def download_blob_from_url(
219221
'BlobQueryError',
220222
'DelimitedJsonDialect',
221223
'DelimitedTextDialect',
224+
'ArrowDialect',
225+
'ArrowType',
222226
'BlobQueryReader',
223227
'ObjectReplicationPolicy',
224228
'ObjectReplicationRule'

sdk/storage/azure-storage-blob/azure/storage/blob/_blob_client.py

Lines changed: 39 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -476,7 +476,7 @@ def upload_blob( # pylint: disable=too-many-locals
476476
and act according to the condition specified by the `match_condition` parameter.
477477
:keyword ~azure.core.MatchConditions match_condition:
478478
The match condition to use upon the etag.
479-
:keyword str if_tags_match_condition
479+
:keyword str if_tags_match_condition:
480480
Specify a SQL where clause on blob tags to operate only on blob with a matching value.
481481
eg. "\"tagname\"='my tag'"
482482
@@ -576,7 +576,7 @@ def _download_blob_options(self, offset=None, length=None, **kwargs):
576576
'lease_access_conditions': access_conditions,
577577
'modified_access_conditions': mod_conditions,
578578
'cpk_info': cpk_info,
579-
'cls': deserialize_blob_stream,
579+
'cls': kwargs.pop('cls', None) or deserialize_blob_stream,
580580
'max_concurrency':kwargs.pop('max_concurrency', 1),
581581
'encoding': kwargs.pop('encoding', None),
582582
'timeout': kwargs.pop('timeout', None),
@@ -636,7 +636,7 @@ def download_blob(self, offset=None, length=None, **kwargs):
636636
and act according to the condition specified by the `match_condition` parameter.
637637
:keyword ~azure.core.MatchConditions match_condition:
638638
The match condition to use upon the etag.
639-
:keyword str if_tags_match_condition
639+
:keyword str if_tags_match_condition:
640640
Specify a SQL where clause on blob tags to operate only on blob with a matching value.
641641
eg. "\"tagname\"='my tag'"
642642
@@ -682,13 +682,19 @@ def _quick_query_options(self, query_expression,
682682
try:
683683
delimiter = input_format.lineterminator
684684
except AttributeError:
685-
delimiter = input_format.delimiter
685+
try:
686+
delimiter = input_format.delimiter
687+
except AttributeError:
688+
raise ValueError("The Type of blob_format can only be DelimitedTextDialect or DelimitedJsonDialect")
686689
output_format = kwargs.pop('output_format', None)
687690
if output_format:
688691
try:
689692
delimiter = output_format.lineterminator
690693
except AttributeError:
691-
delimiter = output_format.delimiter
694+
try:
695+
delimiter = output_format.delimiter
696+
except AttributeError:
697+
pass
692698
else:
693699
output_format = input_format
694700
query_request = QueryRequest(
@@ -729,7 +735,7 @@ def query_blob(self, query_expression, **kwargs):
729735
730736
:param str query_expression:
731737
Required. a query statement.
732-
:keyword Callable[Exception] on_error:
738+
:keyword Callable[~azure.storage.blob.BlobQueryError] on_error:
733739
A function to be called on any processing errors returned by the service.
734740
:keyword blob_format:
735741
Optional. Defines the serialization of the data currently stored in the blob. The default is to
@@ -740,7 +746,8 @@ def query_blob(self, query_expression, **kwargs):
740746
Optional. Defines the output serialization for the data stream. By default the data will be returned
741747
as it is represented in the blob. By providing an output format, the blob data will be reformatted
742748
according to that profile. This value can be a DelimitedTextDialect or a DelimitedJsonDialect.
743-
:paramtype output_format: ~azure.storage.blob.DelimitedTextDialect or ~azure.storage.blob.DelimitedJsonDialect
749+
:paramtype output_format: ~azure.storage.blob.DelimitedTextDialect, ~azure.storage.blob.DelimitedJsonDialect
750+
or list[~azure.storage.blob.ArrowDialect]
744751
:keyword lease:
745752
Required if the blob has an active lease. Value can be a BlobLeaseClient object
746753
or the lease ID as a string.
@@ -762,7 +769,7 @@ def query_blob(self, query_expression, **kwargs):
762769
and act according to the condition specified by the `match_condition` parameter.
763770
:keyword ~azure.core.MatchConditions match_condition:
764771
The match condition to use upon the etag.
765-
:keyword str if_tags_match_condition
772+
:keyword str if_tags_match_condition:
766773
Specify a SQL where clause on blob tags to operate only on blob with a matching value.
767774
eg. "\"tagname\"='my tag'"
768775
@@ -879,7 +886,7 @@ def delete_blob(self, delete_snapshots=False, **kwargs):
879886
and act according to the condition specified by the `match_condition` parameter.
880887
:keyword ~azure.core.MatchConditions match_condition:
881888
The match condition to use upon the etag.
882-
:keyword str if_tags_match_condition
889+
:keyword str if_tags_match_condition:
883890
Specify a SQL where clause on blob tags to operate only on blob with a matching value.
884891
eg. "\"tagname\"='my tag'"
885892
@@ -989,7 +996,7 @@ def get_blob_properties(self, **kwargs):
989996
and act according to the condition specified by the `match_condition` parameter.
990997
:keyword ~azure.core.MatchConditions match_condition:
991998
The match condition to use upon the etag.
992-
:keyword str if_tags_match_condition
999+
:keyword str if_tags_match_condition:
9931000
Specify a SQL where clause on blob tags to operate only on blob with a matching value.
9941001
eg. "\"tagname\"='my tag'"
9951002
@@ -1031,14 +1038,15 @@ def get_blob_properties(self, **kwargs):
10311038
snapshot=self.snapshot,
10321039
lease_access_conditions=access_conditions,
10331040
modified_access_conditions=mod_conditions,
1034-
cls=deserialize_blob_properties,
1041+
cls=kwargs.pop('cls', None) or deserialize_blob_properties,
10351042
cpk_info=cpk_info,
10361043
**kwargs)
10371044
except StorageErrorException as error:
10381045
process_storage_error(error)
10391046
blob_props.name = self.blob_name
1040-
blob_props.snapshot = self.snapshot
1041-
blob_props.container = self.container_name
1047+
if isinstance(blob_props, BlobProperties):
1048+
blob_props.container = self.container_name
1049+
blob_props.snapshot = self.snapshot
10421050
return blob_props # type: ignore
10431051

10441052
def _set_http_headers_options(self, content_settings=None, **kwargs):
@@ -1095,7 +1103,7 @@ def set_http_headers(self, content_settings=None, **kwargs):
10951103
and act according to the condition specified by the `match_condition` parameter.
10961104
:keyword ~azure.core.MatchConditions match_condition:
10971105
The match condition to use upon the etag.
1098-
:keyword str if_tags_match_condition
1106+
:keyword str if_tags_match_condition:
10991107
Specify a SQL where clause on blob tags to operate only on blob with a matching value.
11001108
eg. "\"tagname\"='my tag'"
11011109
@@ -1169,7 +1177,7 @@ def set_blob_metadata(self, metadata=None, **kwargs):
11691177
and act according to the condition specified by the `match_condition` parameter.
11701178
:keyword ~azure.core.MatchConditions match_condition:
11711179
The match condition to use upon the etag.
1172-
:keyword str if_tags_match_condition
1180+
:keyword str if_tags_match_condition:
11731181
Specify a SQL where clause on blob tags to operate only on blob with a matching value.
11741182
eg. "\"tagname\"='my tag'"
11751183
@@ -1516,7 +1524,7 @@ def create_snapshot(self, metadata=None, **kwargs):
15161524
and act according to the condition specified by the `match_condition` parameter.
15171525
:keyword ~azure.core.MatchConditions match_condition:
15181526
The match condition to use upon the etag.
1519-
:keyword str if_tags_match_condition
1527+
:keyword str if_tags_match_condition:
15201528
Specify a SQL where clause on blob tags to operate only on destination blob with a matching value.
15211529
15221530
.. versionadded:: 12.4.0
@@ -1834,7 +1842,7 @@ def acquire_lease(self, lease_duration=-1, lease_id=None, **kwargs):
18341842
and act according to the condition specified by the `match_condition` parameter.
18351843
:keyword ~azure.core.MatchConditions match_condition:
18361844
The match condition to use upon the etag.
1837-
:keyword str if_tags_match_condition
1845+
:keyword str if_tags_match_condition:
18381846
Specify a SQL where clause on blob tags to operate only on blob with a matching value.
18391847
eg. "\"tagname\"='my tag'"
18401848
@@ -1882,7 +1890,7 @@ def set_standard_blob_tier(self, standard_blob_tier, **kwargs):
18821890
18831891
.. versionadded:: 12.4.0
18841892
This keyword argument was introduced in API version '2019-12-12'.
1885-
:keyword str if_tags_match_condition
1893+
:keyword str if_tags_match_condition:
18861894
Specify a SQL where clause on blob tags to operate only on blob with a matching value.
18871895
eg. "\"tagname\"='my tag'"
18881896
@@ -2141,7 +2149,7 @@ def get_block_list(self, block_list_type="committed", **kwargs):
21412149
Required if the blob has an active lease. Value can be a BlobLeaseClient object
21422150
or the lease ID as a string.
21432151
:paramtype lease: ~azure.storage.blob.BlobLeaseClient or str
2144-
:keyword str if_tags_match_condition
2152+
:keyword str if_tags_match_condition:
21452153
Specify a SQL where clause on blob tags to operate only on destination blob with a matching value.
21462154
21472155
.. versionadded:: 12.4.0
@@ -2287,7 +2295,7 @@ def commit_block_list( # type: ignore
22872295
and act according to the condition specified by the `match_condition` parameter.
22882296
:keyword ~azure.core.MatchConditions match_condition:
22892297
The match condition to use upon the etag.
2290-
:keyword str if_tags_match_condition
2298+
:keyword str if_tags_match_condition:
22912299
Specify a SQL where clause on blob tags to operate only on destination blob with a matching value.
22922300
22932301
.. versionadded:: 12.4.0
@@ -2333,7 +2341,7 @@ def set_premium_page_blob_tier(self, premium_page_blob_tier, **kwargs):
23332341
blob and number of allowed IOPS. This is only applicable to page blobs on
23342342
premium storage accounts.
23352343
:type premium_page_blob_tier: ~azure.storage.blob.PremiumPageBlobTier
2336-
:keyword str if_tags_match_condition
2344+
:keyword str if_tags_match_condition:
23372345
Specify a SQL where clause on blob tags to operate only on blob with a matching value.
23382346
eg. "\"tagname\"='my tag'"
23392347
@@ -2402,7 +2410,7 @@ def set_blob_tags(self, tags=None, **kwargs):
24022410
bitflips on the wire if using http instead of https, as https (the default),
24032411
will already validate. Note that this MD5 hash is not stored with the
24042412
blob.
2405-
:keyword str if_tags_match_condition
2413+
:keyword str if_tags_match_condition:
24062414
Specify a SQL where clause on blob tags to operate only on destination blob with a matching value.
24072415
:keyword int timeout:
24082416
The timeout parameter is expressed in seconds.
@@ -2438,7 +2446,7 @@ def get_blob_tags(self, **kwargs):
24382446
:keyword str version_id:
24392447
The version id parameter is an opaque DateTime
24402448
value that, when present, specifies the version of the blob to add tags to.
2441-
:keyword str if_tags_match_condition
2449+
:keyword str if_tags_match_condition:
24422450
Specify a SQL where clause on blob tags to operate only on destination blob with a matching value.
24432451
:keyword int timeout:
24442452
The timeout parameter is expressed in seconds.
@@ -2535,7 +2543,7 @@ def get_page_ranges( # type: ignore
25352543
and act according to the condition specified by the `match_condition` parameter.
25362544
:keyword ~azure.core.MatchConditions match_condition:
25372545
The match condition to use upon the etag.
2538-
:keyword str if_tags_match_condition
2546+
:keyword str if_tags_match_condition:
25392547
Specify a SQL where clause on blob tags to operate only on blob with a matching value.
25402548
eg. "\"tagname\"='my tag'"
25412549
@@ -2684,7 +2692,7 @@ def set_sequence_number(self, sequence_number_action, sequence_number=None, **kw
26842692
and act according to the condition specified by the `match_condition` parameter.
26852693
:keyword ~azure.core.MatchConditions match_condition:
26862694
The match condition to use upon the etag.
2687-
:keyword str if_tags_match_condition
2695+
:keyword str if_tags_match_condition:
26882696
Specify a SQL where clause on blob tags to operate only on blob with a matching value.
26892697
eg. "\"tagname\"='my tag'"
26902698
@@ -2758,7 +2766,7 @@ def resize_blob(self, size, **kwargs):
27582766
and act according to the condition specified by the `match_condition` parameter.
27592767
:keyword ~azure.core.MatchConditions match_condition:
27602768
The match condition to use upon the etag.
2761-
:keyword str if_tags_match_condition
2769+
:keyword str if_tags_match_condition:
27622770
Specify a SQL where clause on blob tags to operate only on blob with a matching value.
27632771
eg. "\"tagname\"='my tag'"
27642772
@@ -2888,7 +2896,7 @@ def upload_page( # type: ignore
28882896
and act according to the condition specified by the `match_condition` parameter.
28892897
:keyword ~azure.core.MatchConditions match_condition:
28902898
The match condition to use upon the etag.
2891-
:keyword str if_tags_match_condition
2899+
:keyword str if_tags_match_condition:
28922900
Specify a SQL where clause on blob tags to operate only on blob with a matching value.
28932901
eg. "\"tagname\"='my tag'"
28942902
@@ -3060,7 +3068,7 @@ def upload_pages_from_url(self, source_url, # type: str
30603068
and act according to the condition specified by the `match_condition` parameter.
30613069
:keyword ~azure.core.MatchConditions match_condition:
30623070
The destination match condition to use upon the etag.
3063-
:keyword str if_tags_match_condition
3071+
:keyword str if_tags_match_condition:
30643072
Specify a SQL where clause on blob tags to operate only on blob with a matching value.
30653073
eg. "\"tagname\"='my tag'"
30663074
@@ -3177,7 +3185,7 @@ def clear_page(self, offset, length, **kwargs):
31773185
and act according to the condition specified by the `match_condition` parameter.
31783186
:keyword ~azure.core.MatchConditions match_condition:
31793187
The match condition to use upon the etag.
3180-
:keyword str if_tags_match_condition
3188+
:keyword str if_tags_match_condition:
31813189
Specify a SQL where clause on blob tags to operate only on blob with a matching value.
31823190
eg. "\"tagname\"='my tag'"
31833191
@@ -3307,7 +3315,7 @@ def append_block( # type: ignore
33073315
and act according to the condition specified by the `match_condition` parameter.
33083316
:keyword ~azure.core.MatchConditions match_condition:
33093317
The match condition to use upon the etag.
3310-
:keyword str if_tags_match_condition
3318+
:keyword str if_tags_match_condition:
33113319
Specify a SQL where clause on blob tags to operate only on blob with a matching value.
33123320
eg. "\"tagname\"='my tag'"
33133321
@@ -3454,7 +3462,7 @@ def append_block_from_url(self, copy_source_url, # type: str
34543462
and act according to the condition specified by the `match_condition` parameter.
34553463
:keyword ~azure.core.MatchConditions match_condition:
34563464
The destination match condition to use upon the etag.
3457-
:keyword str if_tags_match_condition
3465+
:keyword str if_tags_match_condition:
34583466
Specify a SQL where clause on blob tags to operate only on blob with a matching value.
34593467
eg. "\"tagname\"='my tag'"
34603468

sdk/storage/azure-storage-blob/azure/storage/blob/_blob_service_client.py

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -383,6 +383,10 @@ def list_containers(
383383
:param bool include_metadata:
384384
Specifies that container metadata to be returned in the response.
385385
The default value is `False`.
386+
:keyword bool include_deleted:
387+
Specifies that deleted containers to be returned in the response. This is for container restore enabled
388+
account. The default value is `False`.
389+
.. versionadded:: 12.4.0
386390
:keyword int results_per_page:
387391
The maximum number of container names to retrieve per API
388392
call. If the request does not specify the server will return up to 5,000 items.
@@ -401,6 +405,9 @@ def list_containers(
401405
:caption: Listing the containers in the blob service.
402406
"""
403407
include = ['metadata'] if include_metadata else []
408+
include_deleted = kwargs.pop('include_deleted', None)
409+
if include_deleted:
410+
include.append("deleted")
404411

405412
timeout = kwargs.pop('timeout', None)
406413
results_per_page = kwargs.pop('results_per_page', None)
@@ -557,7 +564,7 @@ def delete_container(
557564
**kwargs)
558565

559566
@distributed_trace
560-
def _undelete_container(self, deleted_container_name, deleted_container_version, new_name=None, **kwargs):
567+
def undelete_container(self, deleted_container_name, deleted_container_version, **kwargs):
561568
# type: (str, str, str, **Any) -> ContainerClient
562569
"""Restores soft-deleted container.
563570
@@ -571,12 +578,14 @@ def _undelete_container(self, deleted_container_name, deleted_container_version,
571578
Specifies the name of the deleted container to restore.
572579
:param str deleted_container_version:
573580
Specifies the version of the deleted container to restore.
574-
:param str new_name:
581+
:keyword str new_name:
575582
The new name for the deleted container to be restored to.
583+
If not specified deleted_container_name will be used as the restored container name.
576584
:keyword int timeout:
577585
The timeout parameter is expressed in seconds.
578586
:rtype: ~azure.storage.blob.ContainerClient
579587
"""
588+
new_name = kwargs.pop('new_name', None)
580589
container = self.get_container_client(new_name or deleted_container_name)
581590
try:
582591
container._client.container.restore(deleted_container_name=deleted_container_name, # pylint: disable = protected-access

0 commit comments

Comments
 (0)