I have a vendor that sucks donkey balls. Their systems break often. An endpoint we rely on will start returning [] and take months to fix. They’ll change a data label in their backend and not notice that it flows into all of their filters and stuff.

I have some alerts when my consumers break, but I think I’d like something more direct. What’s the best way to monitor an external API?

I’m imagining some very basic ML that can pop up and tell me that something has changed, like there are more hosts or categories or whatever than usual, that a structure has gone blank or is missing, that some field has gone to 0 or null across the structure. Heck, that a field name has changed.

Is the best way to basically write tests for everything I can think of, and add more as things break, or is there a better tool? I see API monitoring tools but they are for calculating availability for your own APIs, not for enforcing someone else’s!

  • ThirdConsul@lemmy.ml
    10·
    19 hours ago

    So to sum it up:

    • API breaks responses only for new data
    • API does not provide any metadata, like versioning
    • API doesn’t host it’s spec
    • the problem isn’t that the API mutates, but that it starts returning garbage for new data, but not historical.

    You’re out of luck. You can’t prevent it. You can’t foresee it, unless you know beforehand what you’ll call the API with and you can pre-flight it and detect it earlier.

    • Clay_pidgin@sh.itjust.worksOPEnglish
      3·
      19 hours ago

      Luckily the only thing I ever pass it are dates and host IDs. I can check a known input against the known response, but my problem so far hasn’t been them breaking their database but breaking the new data being added or the API itself.

      Yeah, it’s a tough one.

      • ThirdConsul@lemmy.ml
        3·
        18 hours ago

        Preflight it? If you ask external API every 6 hours about known range of host IDs with a date, then 1h before you need that information call the external API and check if it works or returns garbage? That way you can get some extra time to maybe react earlier to an incident? It honestly depends on the nature of your job and the qualities of your traffic, but generally speaking the problem you have is unfixable and the best you can hope for is early detection (if that matters for you).

        If however you’re a pass-through API to the external one, eg. a different service calls your API with a hostID and the hostIDs are not a known finite pool, then you can forget about preflighting.