DKIM, DMARC, SPF, rDNS/PTR, EHLO… all correct and verified in the email “original source,” BY GOOGLE. Still goes to SPAM folder because **** you, apparently. I hate Google.
I put dog shit in my ass and Google knew five minutes before it happened, filmed it, and sold it back to me as a custom highlight of my home alone shenanigans. I find this a bit egregious, personally.
Running your own email server is also a good way to get a ton of headaches from your isp
Wrong. If you aren’t up to the task of hosting email that’s fine but discouraging others from trying is the opposite direction we should be going.
I stopped thinking of any email as “private” regardless of where it’s hosted, when I realized that everyone I email uses Gmail anyways; so all my emails end up going to Google anyways.
It made me an email nihilist
I hate it. it was supposed to be open standard
Microsoft Exchange and the web-based Outlook probably still has a plurality of email volume, because of their dominance in enterprise. Even on the web based consumer space, Apple and Yahoo still have significant market share.
Me and a couple other people have ran into on the internet have thought that we need to invent “email 2”, a clean slate implementation of email that serves it’s asynchronous long form and attachment friendly function while leaving behind Decades of Cruft . We can make it into an encrypted by default and have conveniences that we never had before. It might even be an actual useful thing instead of just being the thing you sign up for accounts with to get to 2fa codes and spam from websites you buy stuff from
Frankly, I’m rather shocked it hasn’t happened by now.
We’ve tried to bandaid over it with various weird things, but fundamentally SMTP is just weird.
I could easily imagine a domain having some SRV record to advertise “hey, I support NeoSMTP”, and peers switching to that instead, possibly pinning on successful NeoSMTP negotiation.
Get rid of a lot of the relaying which has not been useful in modern times and only a liability, and perhaps offer an https option to be less likely to be blocked (a TXT record to indicate url, and a node could declare their NeoSMTP entry point as https://mail.server/NeoSMTP/ or whatever works for them).
Oh, there have been plenty. The issue though is the classic “make a better standard, and now you have N+1 standards”.
Everything is wrong with SMTP. It’s still in use because it’s supported everywhere. Anything that wants to replace it either needs to entirely replace it in one fell swoop, or it needs to be backwards compatible.The result is that occasionally a new thing comes up, a handful of people try it but there isn’t critical mass to make other features really useful so it fades.
Hence my references to DNS records in addition to MX to determine if an endpoint is SMTP only or better. For compatible interop.
Jmap is also an emerging standard. https://jmap.io/
Looks like replacing IMAP rather than SMTP, which is significantly less thorny.
Love this idea!
I feel like Arcane/Delta Chat is a step toward this. It somehow uses an email protocol and it works just like Signal or any other message app! I dunno how the heck it works but it’s super cool.
That’s something I want too: An open standard my friends and family can be on, where “the market” isn’t.
Edit: I see, it seems the issue in the main thread is:
- Gmail blocks custom servers as spam 99% of the time.
- 99% of normieverse uses Gmail. :(
But getting people on other email providers to talk to each other… shouldn’t be an issue then?
“normieverse” hit me like a brick
I have this problem for a while as well. Similar to you all mechanisms are implemented multiple sites say the configuration is correct, even headers in Gmail say DKIM and SPF passed. E-mail arrives everywhere except for gmail.
Thanks for posting, at least I know I’m not the only one.
Tbh, out of all services I’ve seen people self-host, email is the most common one I’ve seen people describe as a bad idea™️.
I just chose a paid mail service (mailbox) and called it a day.
I do not understand all the people who are too afraid to try email hosting on their own that are out here discouraging people cause they want to sound right.
Email is hard cause people don’t host it enough and the few corps ignore the minority problems.
If more people shift back to hosting email the balance will tip again.
It is fine if you don’t want to host email but telling people not too without even trying yourself is anti-selfhost IMHO.
I’ve been hosting my e-mail for over 2 decades now, e-mail goes everywhere but gmail (despite working with them for years). They are doing it exactly so people stop doing it.
Same I use commercial for most of my usages, but I have one properly registered for my selfhosted services.
Same. It’s great for a technical challenge. Don’t do it for your primary email unless you’re ok losing emails and having them rejected a lot.
Yeah we should just let them win. Why fight? It’s not like most of the spam and phishing come from Gmail and hotmail accounts. It’s those fucking self hosters
How’s your self hosted email going? Any advice you can share to avoid some of the major issues?
I’ve run a mail server since the late 90s. I gave up running my own outbound years ago. Whether it gets received is too inconsistent. Primarily, make sure the IP you use isn’t on any spam lists. If it is, get a new IP or try to submit appeals. Make sure you have the standard anti spam/authenticity measures implemented (they’re listed in the OP).
These are the minimum. Years ago it was enough to have reasonable delivery. Unfortunately these aren’t that helpful anymore because spammers also set them up (often with seemingly randomly generated domain names). I assume because not having them was a fairly strong signal for being a spammer, so now they also set them up, which also means there’s not really any configuration / setup you can do to seem legitimate (aside from mail volume and the content itself which will be analyzed).
Anyway running your own SMTP itself is easy. The problem is getting your mail to the inbox or at least spam/junk folder (your mail may get blackholed and not even show up in spam). At least when you get a rejection, you know something is wrong, but many times the receiving server will indicate a success code and simply not deliver it to the user (not even to their spam).
Interesting! You said you gave up on the outbound, does that mean you can’t reply to mails you receive? But if you can reply, does that mean the sender (let’s assume someone with gmail) will get it or would even that likely end up blackholed or in spam?
I can reply yo emails. It’s what jaybone said – I use a 3rd party outbound (SMTP).
I use Thunderbird as my mail client. In there you can define multiple IMAP/pop accounts (inbound) and multiple SMTP (outbound). So when I click reply, the SMTP I have set for that email address is a 3rd party service that sends emails using my domain.
This does have a privacy implication, so whether that’s appropriate for you is for you to decide. The 3rd party can read and save a copy of your outbound email inclusive of any content included with it (like the email you’re replying to if you include it).
Ah nice, that is a great idea. I will try this out, thanks!
What 3rd party service do you use?
Don’t take this as an endorsement as it’s not really intended to be used for this purpose and I’m probably going to switch, but I’ve been using sendgrid for a few years as I was familiar with it from business use. I was using mandrill before that until they changed their pricing and some other things.
EDIT: also worth mentioning I have a grandfathered free plan.
Maybe they send emails using someone else’s smtp server? (Like some paid service that provides smtp?)
@toxicmode @galoisghost I think those of us who can should selfhost an email address at least to contact other selfhosters even if we can’t use it effectively to contact users of Big Email. That’s how the network effect begins
Think of it like how a lot of the Fedi defederated from Gab or/and Thereads, except that this time it’s Big email that’s forcing the defederation
https://badrihippo.thekambattu.rocks/clog/staying-away-from-big-email/
Stop tempting me! I might do it.
One question, I currently use proton mail to use my domain for email and there are no issues afaik. Do you think a few years of using proton will have “whitelisted” my domain enough or will all the issues pop up once I change to hosting it myself?
Maybe the way to fight GMail’s outsized power is to stop using GMail. You can do that without going all the way to self-hosting.
Maybe the way to fight GMail’s outsized power is to stop using GMail.
Stopping your own gmail use is trivially easy. It just scratches the surface. In order to not lick Google’s boots, you also need to stop sending email to gmail users. That means doing an MX lookup before emailing to see if their vanity email address is on a Gmail host. And of course this also means not sharing an email address with gmail users.
I do that, in fact. I also boycott Microsoft. In the end, this means I am not using email much at all.
Yeah, I figure I did enough of a step by moving to Tutanota. Selfhosting email seems like overkill unless you’re running some large organization or doing self-hosting as a hobby.
Honestly self hosted email can be just for self hosters talking to one another and that’s fine
Even using a reputable hosting provider and personal domain will get you spam black holed these days
I wonder if there should be some kind of law or regulation around this. It’s an antitrust issue. Though no one in Congress would understand what this is.
There most certainly should. But Americans live in a corporatocracy.
Does gmail not do this kind of blocking outside of the US?
This here. My emails end up in gmail spam folders occasionally from what I am told. And I don’t self-host. I just use a good webhoster and my own domain. Fuck google.
Many hosting providers won’t even let you try, they don’t want to host potential spammers.
Soon not using a chromium browser will probably be considered a bad idea
I get what you mean, but this website specifically works completely fine on my Firefox (with uBlock). Please check your browser settings for things like disabled JavaScript.
Yeah everything is disabled its Firefox focus, everything always worked fine then like this past week all sorts of sites started showing that
That screen is from cloudflare. It means you’re blocking their JavaScript or iframe elements used to create the bot challenge. I get it quite often as I still use the old uMatrix extension. Temporarily unblocking and refreshing the page will let you through.
OK I went into the settings and disable JavaScript wasn’t active but I tried disabling all the tracker blocking and it worked
I use a free outbound smtp relay that lets me do like 5k emails on the free tier. I use it primarily for alerts and Immich updates / new user links. But I don’t do inbound so don’t have to expose any ports. Only time I did was to do validation with the relay so just a few minutes during setup.
If you want you can PM me your dkim and dmarc and SPF policy. Or just send me your url and I’ll look it up. I used to fix these for a living. It’s likely an issue with your SPF not being properly annotated.
Do you have any go to guides or even an overview of how you generally diagnose email problems would be awesome!
Just reading the bounce backs and spam errors when sending to Gmail and other providers.
That’s cool of you to offer that to them.
They claim its for spam prevention which is why most ISPs actually also block outgoing SMTP port 25.
The dumb thing is, all it really does is push spammers onto using botted/stolen accounts which work way more effectively, and everyone else is forced to set up their email with Google or Microsoft.
If your domain isn’t big enough to be recognized, all that SPF and DMARC won’t mean crap to them.
They claim its for spam prevention which is why most ISPs actually also block outgoing SMTP port 25.
European ISPs block egress port 25, not American ones, generally. Not sure about the rest of the world.
In Europe, one refuge is to send using the GSM networks, which tend not to block egress 25 for some reason.
not American ones, generally.
Well then screw you AT&T lmao
AT&T is the worst of the worst. And I boycott them for countless reasons like snooping on their own customers voluntarily without a warrant. It’s really a hard-right corp. In any case, never tried them so I didn’t know they blocked egress 25.
One trick that works if truth-in-advertising laws are in force: ask the sales people before subscribing if the block port 25. They always say “no, we block nothing” (in my experience). So you subscribe and sign the contract. Then when you see they actually block 25, you have a false advertising situation and also a contract violation (if either verbal contracts are enforcable or if you can get it in writing that nothing is blocked). So you complain. In my experience, they give a gratis upgrade to an enterprise level of service that generally has a static IP and no blocks – for the price of the residential plan you signed up for.
i never had problems with Google, but Outlook always blocks my emails, just because it’s not a standard .com .net etc domain…
I work for state government and even we have trouble with email reputation issues from the big orgs. It’s a pain for everyone.
This is done on purpose, to make people to switch to their e-mail or use them for hosting.
Google did the same thing with XMPP (they were welcomed when they joined, then they made it so XMPP users couldn’t message GTalk/Hangouts users, but the other way worked), they tried similar thing with Usenet, but they never get dominance there so they let their services to be major source of spam on Usenet until they were blocked, ultimately removing Usenet indexing from Google).
I would really encourage to contact our local reps (both federal as well as state) and encourage them to introduce legislation that would punish this practice.
Self-hosting email is quite the undertaking… I don’t have the energy to deal with it and just pay 10€/year for a hosting service.
Here’s an informative (and funny) talk on why self-hosting email in its current state is probably a bad idea.
@laurenceOfSuburbia
This ain’t anything new. They have been doing this for decades and will still do it forever. Just move on with it, since it like trying to figure out something which has no meaning at all.
Just know for sure that every company has systems designed by some developer based on a whimsical mindset of someone at the very top. So, I can care less for these stupid companies and their useless management.It’s basically the same thing Google did with XMPP.
Google Groups allows spammers to subscribe you to their groups in a workspace account, but doesn’t allow you to unsubscribe since you’re not part of that workspace account… Worse - all autoresponders for the spam will also be forwarded to all members of said group.
Since I do not participate in Google Groups, the nuclear option is to file all mails from Google Groups as spam with a Sieve script.
Note the
requirepart may be too large as may sieve script also contains more rules than this one.require ["body", "date", "editheader", "envelope", "fileinto", "imap4flags", "mailbox", "regex", "reject", "variables"]; if anyof( header :contains "List-Subscribe" "groups.google.com", header :matches "X-Google-Group-Id" "*" ) { if header :matches "Subject" "*" { set "subject" "${1}"; } else { set "subject" ""; } deleteheader "Subject"; addheader :last "Subject" "[Google-Groups-Spam] ${subject}"; fileinto "Junk"; }autoresponders for the spam will also be forwarded to all members of said group.
This sounds like a pretty effective way to get yourself unsubscribed from groups you don’t want to be in.
They will be drowned in hundreds of „Thanks for your email, but we don’t do support via mail. Please open a ticket at $URL“ and „Thanks for your mail, we will get back to you shortly. Your ticket ID is #123456“. Also everything you add as an auto response will be distributed to all other victims in that group, so I chose to not be a dick.
I run my own mailserver in docker and it’s not so hard! There was some difficulty at the start with mail going to spam, but after a little while “warming up” the IP with usage, it works fine. It’s hardest to build an IP’s reputation in the beginning because if your mail goes to spam, it doesn’t typically lead to healthy conversations that can support it’s history.
My trick was to forward all my incoming gmail messages from the account I was replacing into the new server - that way I could receive and reply from the new mailserver if people sent mail to the old one. Once it built up a bit of a history it started working fine, and that took maybe a few conversations over a couple of days.
The thing I have noticed is that you very quickly get destroyed by incoming spam - a more advanced filter is definitely on the list.
One of the problems with self hosting is that a lot of IP addresses have historically been used for spam and still have a bad reputation. If you get assigned one of those IP addresses there’s very little you can do about your mails going to spam.
Another is most ISPs will put your IP in the PBL so unless you use a relay your outbound email is going nowhere. Google fiber was slow on updating their PBL when they rolled out service in my area so I got to send direct for several months before having to use an external relay.
I had a small problem with this, but Linode (others do that, too) allowed me to allocate myself a /56 and that solved it. Since IPv6 is preferred if set up - it’s the protocol that gets used. And so my reputation problems went away.
You can use MXToolbox to check for blacklists on your IP. If you find any, you can submit to each party to get removed. I’ve done this for a previous company and it was super easy.
*fuck
