• 54 Posts
  • 1.27K Comments
Joined 3 years ago
cake
Cake day: September 1st, 2023

help-circle






  • Don’t inference providers also provide APIs for cost checking? I imagine you’d need to have a cost-checking service that cancels any CI jobs running an agent. If possible, I’d chain the CI jobs so that one job creates a token with metadata and triggers a new job which runs the agent. The cost-checking service could then trackspending and call the CI API with a kill command for the job whose API token has crossed the limit.

    Maybe the inference platforms even allow calling webhooks on certain events. You would still need to identify which CI job to kill, hence my suggestion to create tokens per job, that would allow you to target the job using that token.

    Alternatively, you have a second process running in the CI job that pings the inference providers API and kills the agent when a certain condition is met.