I have a vendor that sucks donkey balls. Their systems break often. An endpoint we rely on will start returning [] and take months to fix. They’ll change a data label in their backend and not notice that it flows into all of their filters and stuff.
I have some alerts when my consumers break, but I think I’d like something more direct. What’s the best way to monitor an external API?
I’m imagining some very basic ML that can pop up and tell me that something has changed, like there are more hosts or categories or whatever than usual, that a structure has gone blank or is missing, that some field has gone to 0 or null across the structure. Heck, that a field name has changed.
Is the best way to basically write tests for everything I can think of, and add more as things break, or is there a better tool? I see API monitoring tools but they are for calculating availability for your own APIs, not for enforcing someone else’s!
I made my career out of automated testing with a focus on apis. I’m not aware of any easy tool to do what you want. The easiest way to quick whip up basic api tests that I’ve found is python/pytest with requests. You can parameterize lots of inputs, run tests in parallel, easily add new endpoints as you go, benchmark the apis for response times, etc. It’ll take a lot of work in the beginning, then save you a lot of work in the end.
Now, AI will be able to make the process go faster. If you give it a sample input and output it can do 95% of a pytest in 10s. But beware that last 5%.
Yeah I would use python and pytest, probably.
You need to decide what you expect to be a passing case. Known keys are all there? All values in acceptable range? Do you have anything where you know exactly what the response should be?
How many endpoints are there?