Plenty of us are using Docker, Podman, Incus, chroot jails, etc to isolate services.
It has become good practice and it makes setting up yet another service, usually, so convenient.
Some services like YunoHost, StartOS, Cloudron and others try to facilitate the process.
What I haven’t seen though is a way to facilitate interoperability BETWEEN services we self-host. Sure there are plugins for each service, e.g. https://www.npmjs.com/package/peertube-plugin-livechat to provide XMPP chat for PeerTube, or anecdotal discussions e.g. https://github.com/jitsi/jitsi-meet/issues/7601 to embed PeerTube on Jitsi Meet.
So… how do YOU do it? How do you make on self-hosted service with another? Do you check after each one you install in the plugin category? Do you write your own plugins or extensions? Do you have a design pattern (e.g. Swagger API discovery with token generation per service, “cheat” via sockets, use a dedicate new service or even host) which you repeat?
I do ask because I bet most of you have a moment like this :
- Hey how about we start this new project together?
- Yes, let’s change the World!
- OK let’s write manifesto.md
- Where are we going to host it?
- Hmmm we could use my Cryptpad instance…
- OK but I don’t get notification on my GMail, could we use GoogleDocs instead?
So… I feel like FLOSS self-hosting is honestly on-par functionality-wise with proprietary solutions. I might be bias but it’s rare when I think “Damn… that’s cool, shame I can’t have it at home”. I can nearly always (in fact I have a hard time thinking of an example) self-host functional equivalent solutions myself. The ONE thing that I feel is often missing is integration which relies on interoperability.
How do YOU it?
PS: this isn’t about ntfy, PeerTube, HA or any specific service to a specific problem, it’s about HOW to facilitate, when one wants to, already great services work together.
You must log in or register to comment.
I haven’t tried it yet, but I think you want something like n8n. Yeah, I know they claim to be very AI nowadays, but I had them on my radar before that. It’s essentially an automation platform you can use to glue services together that weren’t intended for each other. https://github.com/n8n-io/n8n
Thanks I’ll dig deeper. I guess I do want something like n8n but ideally :
- no AI
- self hosted FLOSS services first, if possibly only (can just be filters like F-Droid has)
- no need for no-code / low-code, code is great
which makes me wonder what they do provide, e.g. is it mostly indexing existing plugins and then some scaffolding for non coders?
I use authentik, which emables single sign on (the same account) between services.
Authentik is a bit complex and irritating at times, so I would recommend voidauth or kanidm as alternatives for most self hosters.
Thanks, that’s indeed exactly the kind of thing I’m looking for “The authentication glue you need.” but even more generalized than that, e.g. just “the glue you need.” not solely for authentication.
Edit: to clarify and coming back after leaving few other comments, the 1 thing authentik has is that it is a cross-service need, namely nearly all services do need authentication AND, probably consequence of that, there are conventions and standards already in place, e.g. SAML, OAuth2/OIDC, LDAP, Auth0. So that makes everything much easier.
APIs. Or the ends are achieved by sharing data between apps in common data storage. But I prefer to be a tourist in my infrastructure, I no longer hand-bomb changes to systems.
My design pattern is essentially to integrate more and more of the container creation into config. Right now I’m using ansible and it’s nice. More automation means troubleshooting has fewer variables.
I had issues yesterday with a package upgrade across several containers, and it ended up being two config changes. I cycle the apps and done. That’s it.
Indeed and for PeerTube for example it has an API, cf https://docs.joinpeertube.org/api-rest-reference.html which I did use. It also provides SWAGGER so that could facilitate integration with others services also providing APIs. I was starting to think that the meta service could have read only public only token generated for each new service and provide a SWAGGER endpoint to facilitate using the API of more than 1 service.
Hmmm we could use my Cryptpad instance… OK but I don’t get notification on my GMail
😡 ntfy.sh 😡
eh… yes thanks and I do actually have ntfy (I also warmly recommend it) but this was just an example.
It sounds like you’re describing Home Assistant? HA has a ton of integrations into a lot of self-hosted services not just IoT devices.
Well I do have Home Assistant, been running it for years, but HA is solely for … well home assisting (or IoT). HA as integrations but let’s say I want to use HA with … any of my other services, e.g. CopyParty to maybe store logs and makes them available or PeerTube to have videos from my camera, I can look at HA integrations, or CopyParty… issues maybe, or PeerTube npm registry.
My point being that HA is a good example with integrations but it’s just one example. If I do take this example seriously though, is there a mechanism beside manual search in the list of integration that would list integrations with my services directly?
@utopiah @yaroto98 re #HomeAssistant integrations, you can also look at what is available under #HACS library, search or ask for help in any of the community forums for #HA such as https://community.home-assistant.io or on reddit, discord , git hub. It integrates nicely with #homebridge and also with containerized systems e.g., docker, portainer.
Can’t say I’ve run into a need for such consideration yet. Excluding stacks explicitly meant to work together to some degree most of my services are an island to themselves and I like it that way. Then as far as notifications are concerned pretty much every supports at least email or ntfy.sh.
most of my services are an island to themselves
same
and I like it that way.
… well that’s the part I’m challenging. I was thinking like this but I’m wondering if that could be improved.
PS: I use ntfy and like it, that was just an example.
… well that’s the part I’m challenging. I was thinking like this but I’m wondering if that could be improved.
Do you have a specific use case for two containers that you want to talk to each other? There are some Docker containers that will cozy up, but as you point out, there are some that don’t. Maybe it would be worth the time to consult with the dev teams of said containers, and see if they have any suggestions or ways to go about it. It would seem to me that would be your best point to deviate from. Couldn’t hurt. I would create a defined road map of what you are trying to accomplish before hand, and run it by the dev teams. Just give them a little time to respond. They are real people with real lives too.
As I’m thinking about it, I wonder if your solution might be automation? Something like n8n might allow cross container exchanges. Of course, those data points have to be present for you to tap them tho.
Do you have a specific use case for two containers that you want to talk to each other?
Sure, for example once a Jitsi Meet meeting ends (more than 1 person in a room in, everybody gone), save the chat log to CopyParty e.g. WebDAV push to /meetingname_date.txt would be enough to be useful. It’s something we tend to do manually on a regular basis.
road map of what you are trying to accomplish before hand, and run it by the dev teams.
Yes no rush and I can code so I would be able to test before suggesting anything.
As I’m thinking about it, I wonder if your solution might be automation?
I don’t touch AI but I do think conventions, e.g. not “just” an API but SWAGGER, specific filesystem on mountpoints, etc could facilitate this.
@utopiah #YunoHost, #StartOS, #Cloudron, #coopcloud, #uncloud and the people that write the recipies for the applications running on them are the answer.
Modelled solutions that check for related applications as they install are the way that encourage more people to self host by making installation, configuration, backup, update and removal easier.
Thanks, are you saying there is a mechanism in place, e.g. does YunoHost suggests plugins or integrations for services it manages?
My process typically goes: ooh, shiny new service!
If it has a docker compose, I’ll read up a little more on it, then set it up.
At that point, I’ll cry a bit in having to change my nginx config, because it’s new and unsupported, figure out what’s actually needed, then realize I didn’t set it up with authentik or even check if it was supported.
Usually I’ll dig around and someone got something close, so I’ll set it up that way, and if it works, great! If not, dig through the documents.Any further interoperability is luck based.
(I’m currently trying to get element-call working before I open it to my family, the whole matrix stack is leaving me scarred )
Yes I can relate to the process.
Any further interoperability is luck based.
Unfortunately I can relate to that, hence the question here :D
