I have a vendor that sucks donkey balls. Their systems break often. An endpoint we rely on will start returning [] and take months to fix. They’ll change a data label in their backend and not notice that it flows into all of their filters and stuff.
I have some alerts when my consumers break, but I think I’d like something more direct. What’s the best way to monitor an external API?
I’m imagining some very basic ML that can pop up and tell me that something has changed, like there are more hosts or categories or whatever than usual, that a structure has gone blank or is missing, that some field has gone to 0 or null across the structure. Heck, that a field name has changed.
Is the best way to basically write tests for everything I can think of, and add more as things break, or is there a better tool? I see API monitoring tools but they are for calculating availability for your own APIs, not for enforcing someone else’s!
Ask them to generate a schema file that you can download from the api. Or at least an endpoint that returns a hash of the current api schema file. That’s cheap versioning telling you if something changes.
You can always use the swagger schema to verify the api. So ask some basic questions what should always be true and put that into validation scripts. If they use a framework, HEAD requests usually tell you some things.
Last really bad vendor had an openapi page that listed the endpoints but the api wouldn’t adhere to the details given there. I discovered that their website used the api all the time and surfing that i was able to discover which parameters were required etc.
Last idea is statistics. Grab any count data you can get, like from pagination data and create a baseline of available data over time. That gives you an expected count and you can detect significant divergences.
I tend to show up at the vendors it guys in person and bribe them into helping me behind their bosses backs. Chocolate, coffee and some banter can do wonders.
I’m 3,500 miles from the vendor’s devs, sadly.
Asking them to put the swagger file itself behind the API is a good idea. Their dev backlog is 3-24 months.
I used the same trick to determine the required headers and parameters - I checked their website which uses the same API.
The source of their delays is that different devs or teams “own” different endpoints and make their changes without documenting. It’s annoying, stuff like the same data being in field “hostId” on one endpoint but “deviceId” on another.
Just build a few selenium Tests to ensure the API requests the website performs don’t change without you noticing :)
This is why you have requirements which are agreed upon and affect payment if not upheld. If you start being firmer, they might move quicker. 24 month lead team is bullshit.
They have accepted the penalties as the cost of doing business, and the decision makers on my side are worried about opening it up again. It’s a custom hardware + custom software thing so there aren’t that many options!