Skip to content

Latest commit

 

History

History
318 lines (212 loc) · 14.1 KB

call-analyze-image.md

File metadata and controls

318 lines (212 loc) · 14.1 KB
title titleSuffix description manager author ms.author ms.service ms.topic ms.date
Call the Image Analysis 3.2 API
Azure AI services
Learn how to call the Image Analysis API and configure its behavior.
nitinme
PatrickFarley
pafarley
azure-ai-vision
how-to
11/01/2024

Call the Image Analysis 3.2 API

This article demonstrates how to call the Image Analysis version 3.2 API to return information about an image's visual features. It also shows you how to parse the returned information using the client SDKs or REST API.

This guide assumes you've already created a Vision resource and obtained a key and an endpoint URL. If you're using a client SDK, you also need to authenticate a client object. For more information about these steps, see the Image Analysis quickstart.

Submit data to the service

The code in this guide uses remote images referenced by URL. You might want to try different images on your own to see the full capability of the Image Analysis features.

REST

When analyzing a remote image, you specify the image's URL by formatting the request body like this: {"url":"http://example.com/images/test.jpg"}.

To analyze a local image, put the binary image data in the HTTP request body.

C#

In your main class, save a reference to the URL of the image you want to analyze.

[!code-csharp]

To analyze a local image, see the ComputerVisionClient methods, such as AnalyzeImageInStreamAsync. Or, see the sample code on GitHub for scenarios involving local images.

Java

In your main class, save a reference to the URL of the image you want to analyze.

[!code-java]

To analyze a local image, see the ComputerVision methods, such as AnalyzeImage. Or, see the sample code on GitHub for scenarios involving local images.

JavaScript

In your main function, save a reference to the URL of the image you want to analyze.

[!code-javascript]

To analyze a local image, see the ComputerVisionClient methods, such as describeImageInStream. Or, see the sample code on GitHub for scenarios involving local images.

Python

Save a reference to the URL of the image you want to analyze.

[!code-python]

To analyze a local image, see the ComputerVisionClientOperationsMixin methods, such as analyze_image_in_stream. Or, see the sample code on GitHub for scenarios involving local images.

Determine how to process the data

Select visual features

The Analyze API gives you access to all of the service's image analysis features. Choose which operations to do based on your own use case. For a description of each feature, see the Azure AI Vision overview. The examples in the following sections add all of the available visual features, but for practical usage you likely only need one or two.

REST

You can specify which features you want to use by setting the URL query parameters of the Analyze API. A parameter can have multiple values, separated by commas. Each feature you specify requires more computation time, so only specify what you need.

URL parameter Value Description
features Read reads the visible text in the image and outputs it as structured JSON data
features Description describes the image content with a complete sentence in supported languages
features SmartCrops finds the rectangle coordinates that would crop the image to a desired aspect ratio while preserving the area of interest
features Objects detects various objects within an image, including the approximate location. The Objects argument is only available in English
features Tags tags the image with a detailed list of words related to the image content

A populated URL might look like this:

<endpoint>/vision/v3.2/analyze?visualFeatures=Tags

C#

Define your new method for image analysis. Add the following code, which specifies visual features you'd like to extract in your analysis. See the VisualFeatureTypes enum for a complete list.

[!code-csharp]

Java

Specify which visual features you'd like to extract in your analysis. See the VisualFeatureTypes enum for a complete list.

[!code-java]

JavaScript

Specify which visual features you'd like to extract in your analysis. See the VisualFeatureTypes enum for a complete list.

[!code-javascript]

Python

Specify which visual features you'd like to extract in your analysis. See the VisualFeatureTypes enum for a complete list.

[!code-python]

Specify languages

You can also specify the language of the returned data.

REST

The following URL query parameter specifies the language. The default value is en.

URL parameter Value Description
language en English
language es Spanish
language ja Japanese
language pt Portuguese
language zh Simplified Chinese

A populated URL might look like this:

<endpoint>/vision/v3.2/analyze?visualFeatures=Tags&language=en

C#

Use the language parameter of the AnalyzeImageAsync call to specify a language.

Language Value
English en
Spanish es
Japanese ja
Portuguese pt
Simplified Chinese zh

A method call that specifies a language might look like the following.

ImageAnalysis results = await client.AnalyzeImageAsync(imageUrl, visualFeatures: features, language: "en");

Java

Use the AnalyzeImageOptionalParameter input in your Analyze call to specify a language.

Language Value
English en
Spanish es
Japanese ja
Portuguese pt
Simplified Chinese zh

A method call that specifies a language might look like the following.

ImageAnalysis analysis = compVisClient.computerVision().analyzeImage().withUrl(pathToRemoteImage)
    .withVisualFeatures(featuresToExtractFromLocalImage)
    .language("en")
    .execute();

JavaScript

Use the language property of the ComputerVisionClientAnalyzeImageOptionalParams input in your Analyze call to specify a language.

Language Value
English en
Spanish es
Japanese ja
Portuguese pt
Simplified Chinese zh

A method call that specifies a language might look like the following.

const result = (await computerVisionClient.analyzeImage(imageURL,{visualFeatures: features, language: 'en'}));

Python

Use the language parameter of your analyze_image call to specify a language.

Language Value
English en
Spanish es
Japanese ja
Portuguese pt
Simplified Chinese zh

A method call that specifies a language might look like the following.

results_remote = computervision_client.analyze_image(remote_image_url , remote_image_features, remote_image_details, 'en')

Get results from the service

This section shows you how to parse the results of the API call. It includes the API call itself.

Note

Scoped API calls

Some of the features in Image Analysis can be called either directly or through the Analyze API call. For example, you can do a scoped analysis of only image tags by making a request to <endpoint>/vision/v3.2/tag (or to the corresponding method in the SDK). See the reference documentation for other features that can be called separately.

REST

The service returns a 200 HTTP response, and the body contains the returned data in the form of a JSON string. The following text is an example of a JSON response.

{
    "metadata":
    {
        "width": 300,
        "height": 200
    },
    "tagsResult":
    {
        "values":
        [
            {
                "name": "grass",
                "confidence": 0.9960499405860901
            },
            {
                "name": "outdoor",
                "confidence": 0.9956876635551453
            },
            {
                "name": "building",
                "confidence": 0.9893627166748047
            },
            {
                "name": "property",
                "confidence": 0.9853052496910095
            },
            {
                "name": "plant",
                "confidence": 0.9791355729103088
            }
        ]
    }
}

Error codes

See the following list of possible errors and their causes:

  • 400
    • InvalidImageUrl - Image URL is badly formatted or not accessible
    • InvalidImageFormat - Input data isn't a valid image
    • InvalidImageSize - Input image is too large
    • NotSupportedVisualFeature - Specified feature type isn't valid
    • NotSupportedImage - Unsupported image, for example child pornography
    • InvalidDetails - Unsupported detail parameter value
    • NotSupportedLanguage - The requested operation isn't supported in the language specified
    • BadArgument - More details are provided in the error message
  • 415 - Unsupported media type error. The Content-Type isn't in the allowed types:
    • For an image URL, Content-Type should be application/json
    • For a binary image data, Content-Type should be application/octet-stream or multipart/form-data
  • 500
    • FailedToProcess
    • Timeout - Image processing timed out
    • InternalServerError

C#

The following code calls the Image Analysis API and prints the results to the console.

[!code-csharp]

Java

The following code calls the Image Analysis API and prints the results to the console.

[!code-java]

JavaScript

The following code calls the Image Analysis API and prints the results to the console.

[!code-javascript]

Python

The following code calls the Image Analysis API and prints the results to the console.

[!code-python]

Tip

While working with Azure AI Vision, you might encounter transient failures caused by rate limits enforced by the service, or other transient problems like network outages. For information about handling these types of failures, see the Retry pattern in the Cloud Design Patterns guide, and the related Circuit Breaker pattern.

Related content