BlazeMeter’s API Monitoring service, which allows users to evaluate the uptime, performance and correctness of an API, had system issues in the US California location. As a result, the API Monitoring tests running from US California were stuck in "working" state.
Root cause of the problem:
One of the network diagnostics tools was missing in the GCP agents in the US California location, which are used by API Monitoring service to send requests to the destinations in case of test failure due to infrastructure or network issues.
Due to the missing tool, the agent was not able to finish any test which was timed out or failed due to network or infrastructure issues. Hence, all the tests in the US California were stuck at “working” state.
Action Taken:
Installed the required network diagnostic tool in the GCP Test agent and deployed it to the US California.
Further findings were done to check why the tool was missing on this GCP location and fixed CI/CD process.
Posted Apr 01, 2020 - 13:19 PDT
Resolved
All locations are now running normally.
Posted Mar 28, 2020 - 10:02 PDT
Identified
Tests running from US California are now completing but with some delay. We are working on a resolution.
Posted Mar 28, 2020 - 06:46 PDT
Investigating
API Monitoring tests run from US California location are stuck in "working" state. We are investigating.
Posted Mar 28, 2020 - 06:02 PDT
This incident affected: API Monitoring (Runscope).