privateca.snippets.test_subordinate_ca: test_subordinate_certificate_authority failed #9081

flaky-bot · 2023-02-03T14:39:21Z

This test failed!

To configure my behavior, see the Flaky Bot documentation.

If I'm commenting on this issue too often, add the flakybot: quiet label and
I will stop commenting.

commit: 46ffcc2
buildURL: Build Status, Sponge
status: failed

Test output

Traceback (most recent call last): File "/workspace/privateca/snippets/.nox/py-3-8/lib/python3.8/site-packages/google/api_core/grpc_helpers.py", line 72, in error_remapped_callable return callable_(*args, **kwargs) File "/workspace/privateca/snippets/.nox/py-3-8/lib/python3.8/site-packages/grpc/_channel.py", line 946, in __call__ return _end_unary_response_blocking(state, call, False, None) File "/workspace/privateca/snippets/.nox/py-3-8/lib/python3.8/site-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking raise _InactiveRpcError(state) grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with: status = StatusCode.RESOURCE_EXHAUSTED details = "Quota exceeded for quota metric 'CreateCertificateAuthority requests' and limit 'CreateCertificateAuthority requests per minute per region' of service 'privateca.googleapis.com' for consumer 'project_number:685750528901'." debug_error_string = "UNKNOWN:Error received from peer ipv4:173.194.193.95:443 {created_time:"2023-02-03T14:39:09.025945768+00:00", grpc_status:8, grpc_message:"Quota exceeded for quota metric \'CreateCertificateAuthority requests\' and limit \'CreateCertificateAuthority requests per minute per region\' of service \'privateca.googleapis.com\' for consumer \'project_number:685750528901\'."}" >

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/workspace/privateca/snippets/test_subordinate_ca.py", line 51, in test_subordinate_certificate_authority
create_subordinate_ca(
File "/workspace/privateca/snippets/create_subordinate_ca.py", line 95, in create_subordinate_ca
operation = ca_service_client.create_certificate_authority(request=request)
File "/workspace/privateca/snippets/.nox/py-3-8/lib/python3.8/site-packages/google/cloud/security/privateca_v1/services/certificate_authority_service/client.py", line 1423, in create_certificate_authority
response = rpc(
File "/workspace/privateca/snippets/.nox/py-3-8/lib/python3.8/site-packages/google/api_core/gapic_v1/method.py", line 113, in call
return wrapped_func(*args, **kwargs)
File "/workspace/privateca/snippets/.nox/py-3-8/lib/python3.8/site-packages/google/api_core/grpc_helpers.py", line 74, in error_remapped_callable
raise exceptions.from_grpc_error(exc) from exc
google.api_core.exceptions.ResourceExhausted: 429 Quota exceeded for quota metric 'CreateCertificateAuthority requests' and limit 'CreateCertificateAuthority requests per minute per region' of service 'privateca.googleapis.com' for consumer 'project_number:685750528901'. [reason: "RATE_LIMIT_EXCEEDED"
domain: "googleapis.com"
metadata {
key: "service"
value: "privateca.googleapis.com"
}
metadata {
key: "quota_metric"
value: "privateca.googleapis.com/create_certificate_authority_requests"
}
metadata {
key: "quota_location"
value: "us-central1"
}
metadata {
key: "quota_limit"
value: "CasPerMinute"
}
metadata {
key: "quota_limit_value"
value: "5"
}
metadata {
key: "consumer"
value: "projects/685750528901"
}
]

The text was updated successfully, but these errors were encountered:

flaky-bot · 2023-02-03T16:44:24Z

Looks like this issue is flaky. 😟

I'm going to leave this open and stop commenting.

A human should fix and close this.

When run at the same commit (46ffcc2), this test passed in one build (Build Status, Sponge) and failed in another build (Build Status, Sponge).

* Fixing flacky privateca tests (#9075, #9081) * Fixing backoff to be longer than 1 minute

flaky-bot · 2023-03-22T15:20:45Z

Oops! Looks like this issue is still flaky. It failed again. 😬

I reopened the issue, but a human will need to close it again.

commit: 63855c9
buildURL: Build Status, Sponge
status: failed

Test output

Traceback (most recent call last):
  File "/workspace/privateca/snippets/.nox/py-3-11/lib/python3.11/site-packages/google/api_core/grpc_helpers.py", line 72, in error_remapped_callable
    return callable_(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/privateca/snippets/.nox/py-3-11/lib/python3.11/site-packages/grpc/_channel.py", line 946, in __call__
    return _end_unary_response_blocking(state, call, False, None)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/privateca/snippets/.nox/py-3-11/lib/python3.11/site-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking
    raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
	status = StatusCode.RESOURCE_EXHAUSTED
	details = "Quota exceeded for quota metric 'CreateCertificateAuthority requests' and limit 'CreateCertificateAuthority requests per minute per region' of service 'privateca.googleapis.com' for consumer 'project_number:173425378622'."
	debug_error_string = "UNKNOWN:Error received from peer ipv4:142.250.1.95:443 {created_time:"2023-03-22T15:20:27.699917624+00:00", grpc_status:8, grpc_message:"Quota exceeded for quota metric \'CreateCertificateAuthority requests\' and limit \'CreateCertificateAuthority requests per minute per region\' of service \'privateca.googleapis.com\' for consumer \'project_number:173425378622\'."}"
>
The above exception was the direct cause of the following exception:
Traceback (most recent call last):

File "/workspace/privateca/snippets/test_subordinate_ca.py", line 58, in test_subordinate_certificate_authority

create_subordinate_ca(

File "/workspace/privateca/snippets/create_subordinate_ca.py", line 95, in create_subordinate_ca

operation = ca_service_client.create_certificate_authority(request=request)

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/workspace/privateca/snippets/.nox/py-3-11/lib/python3.11/site-packages/google/cloud/security/privateca_v1/services/certificate_authority_service/client.py", line 1435, in create_certificate_authority

response = rpc(

^^^^

File "/workspace/privateca/snippets/.nox/py-3-11/lib/python3.11/site-packages/google/api_core/gapic_v1/method.py", line 113, in call

return wrapped_func(*args, **kwargs)

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/workspace/privateca/snippets/.nox/py-3-11/lib/python3.11/site-packages/google/api_core/grpc_helpers.py", line 74, in error_remapped_callable

raise exceptions.from_grpc_error(exc) from exc

google.api_core.exceptions.ResourceExhausted: 429 Quota exceeded for quota metric 'CreateCertificateAuthority requests' and limit 'CreateCertificateAuthority requests per minute per region' of service 'privateca.googleapis.com' for consumer 'project_number:173425378622'. [reason: "RATE_LIMIT_EXCEEDED"

domain: "googleapis.com"

metadata {

key: "service"

value: "privateca.googleapis.com"

}

metadata {

key: "quota_metric"

value: "privateca.googleapis.com/create_certificate_authority_requests"

}

metadata {

key: "quota_location"

value: "us-central1"

}

metadata {

key: "quota_limit"

value: "CasPerMinute"

}

metadata {

key: "quota_limit_value"

value: "5"

}

metadata {

key: "consumer"

value: "projects/173425378622"

}

]

nicain · 2023-03-24T20:24:27Z

Lots of changes on the infrastructure side may have resolved this. Closing to see if this will reoccur.

flaky-bot · 2023-03-24T21:17:58Z

Oops! Looks like this issue is still flaky. It failed again. 😬

I reopened the issue, but a human will need to close it again.

commit: 2f93ea6
buildURL: Build Status, Sponge
status: failed

Test output

Traceback (most recent call last): File "/workspace/privateca/snippets/.nox/py-3-7/lib/python3.7/site-packages/google/api_core/grpc_helpers.py", line 72, in error_remapped_callable return callable_(*args, **kwargs) File "/workspace/privateca/snippets/.nox/py-3-7/lib/python3.7/site-packages/grpc/_channel.py", line 946, in __call__ return _end_unary_response_blocking(state, call, False, None) File "/workspace/privateca/snippets/.nox/py-3-7/lib/python3.7/site-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking raise _InactiveRpcError(state) grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with: status = StatusCode.RESOURCE_EXHAUSTED details = "Quota exceeded for quota metric 'CreateCertificateAuthority requests' and limit 'CreateCertificateAuthority requests per minute per region' of service 'privateca.googleapis.com' for consumer 'project_number:147863112738'." debug_error_string = "UNKNOWN:Error received from peer ipv4:209.85.145.95:443 {created_time:"2023-03-24T21:17:41.025904241+00:00", grpc_status:8, grpc_message:"Quota exceeded for quota metric \'CreateCertificateAuthority requests\' and limit \'CreateCertificateAuthority requests per minute per region\' of service \'privateca.googleapis.com\' for consumer \'project_number:147863112738\'."}" >

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/workspace/privateca/snippets/test_subordinate_ca.py", line 66, in test_subordinate_certificate_authority
CA_DURATION,
File "/workspace/privateca/snippets/create_subordinate_ca.py", line 95, in create_subordinate_ca
operation = ca_service_client.create_certificate_authority(request=request)
File "/workspace/privateca/snippets/.nox/py-3-7/lib/python3.7/site-packages/google/cloud/security/privateca_v1/services/certificate_authority_service/client.py", line 1439, in create_certificate_authority
metadata=metadata,
File "/workspace/privateca/snippets/.nox/py-3-7/lib/python3.7/site-packages/google/api_core/gapic_v1/method.py", line 113, in call
return wrapped_func(*args, **kwargs)
File "/workspace/privateca/snippets/.nox/py-3-7/lib/python3.7/site-packages/google/api_core/grpc_helpers.py", line 74, in error_remapped_callable
raise exceptions.from_grpc_error(exc) from exc
google.api_core.exceptions.ResourceExhausted: 429 Quota exceeded for quota metric 'CreateCertificateAuthority requests' and limit 'CreateCertificateAuthority requests per minute per region' of service 'privateca.googleapis.com' for consumer 'project_number:147863112738'. [reason: "RATE_LIMIT_EXCEEDED"
domain: "googleapis.com"
metadata {
key: "service"
value: "privateca.googleapis.com"
}
metadata {
key: "quota_metric"
value: "privateca.googleapis.com/create_certificate_authority_requests"
}
metadata {
key: "quota_location"
value: "us-central1"
}
metadata {
key: "quota_limit"
value: "CasPerMinute"
}
metadata {
key: "quota_limit_value"
value: "5"
}
metadata {
key: "consumer"
value: "projects/147863112738"
}
]

product-auto-label bot added the samples Issues that are directly related to samples. label Feb 3, 2023

blunderbuss-gcf bot assigned m-strzelczyk Feb 3, 2023

flaky-bot bot added the flakybot: flaky Tells the Flaky Bot not to close or comment on this issue. label Feb 3, 2023

kweinmeister added priority: p2 Moderately-important priority. Fix may not be included in next release. and removed priority: p1 Important issue which blocks shipping the next release. Will be fixed prior to next release. labels Feb 10, 2023

rsamborski added a commit that referenced this issue Feb 28, 2023

Fixing flacky privateca tests (#9075, #9081)

ba57016

rsamborski assigned rsamborski and unassigned m-strzelczyk Feb 28, 2023

rsamborski mentioned this issue Feb 28, 2023

Fixing flacky privateca tests (#9075, #9081) #9189

Merged

2 tasks

leahecole pushed a commit that referenced this issue Mar 1, 2023

Fixing flacky privateca tests (#9075, #9081) (#9189)

e635233

* Fixing flacky privateca tests (#9075, #9081) * Fixing backoff to be longer than 1 minute

telpirion pushed a commit that referenced this issue Mar 13, 2023

Fixing flacky privateca tests (#9075, #9081) (#9189)

6822a00

* Fixing flacky privateca tests (#9075, #9081) * Fixing backoff to be longer than 1 minute

m-strzelczyk mentioned this issue Mar 17, 2023

fix(privateca.tests): Adding a random amount of seconds to backoff timer #9293

Merged

dandhlee closed this as completed Mar 21, 2023

flaky-bot bot reopened this Mar 22, 2023

nicain closed this as completed Mar 24, 2023

flaky-bot bot reopened this Mar 24, 2023

m-strzelczyk closed this as completed in #9293 Mar 29, 2023

flaky-bot bot mentioned this issue May 8, 2023

privateca.snippets.test_subordinate_ca: test_subordinate_certificate_authority failed #9910

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

privateca.snippets.test_subordinate_ca: test_subordinate_certificate_authority failed #9081

privateca.snippets.test_subordinate_ca: test_subordinate_certificate_authority failed #9081

flaky-bot bot commented Feb 3, 2023

flaky-bot bot commented Feb 3, 2023

flaky-bot bot commented Mar 22, 2023

nicain commented Mar 24, 2023

flaky-bot bot commented Mar 24, 2023

privateca.snippets.test_subordinate_ca: test_subordinate_certificate_authority failed #9081

privateca.snippets.test_subordinate_ca: test_subordinate_certificate_authority failed #9081

Comments

flaky-bot bot commented Feb 3, 2023

flaky-bot bot commented Feb 3, 2023

flaky-bot bot commented Mar 22, 2023

nicain commented Mar 24, 2023

flaky-bot bot commented Mar 24, 2023