Slrpnk instance is down till mid July; they might relaunch their server on piefed.
-
Yeah, the xmpp server is down too. That is something that bugs me quite a bit and I will probably move that one to an external small VPS to retain a more secure backup communication channel.
(speaking as a slrpnk user): Another backup communication strategy (once things are backup) might be to designate somewhere on a non-slrpnk instance as a place where people can check for updates if things go down; when I first discovered the outage, I wasn't sure where to go to check for info/updates.
Unrelatedly, I hope that this unexpected outage isn't causing you or other admins too much stress. Whilst the extended nature of this outage is unfortunate, I respect that you're using this as an opportunity to migrate to a more robust solution. This kind of resilience focussed response is a key part of the solarpunk ethos, in my view.
Some people have said that such a long outage seems likely to kill an instance, but for my part, this community is worth waiting for — I have enjoyed having an account on this instance because it feels like the perfect blend of small enough to have a distinct culture and ethos, but is large enough to be robust and diverse.
-
You underestimate the userbase. I made a temp account in the mean time, but we are a hyper tight knit community. We will probably lose accounts - no question - but the core userbase will return
I agree. I replied to Kris elsewhere saying this, but I am super glad to have been a part of this instance because it feels like a nice balance of being large enough to be robust and diverse, but small enough to have a distinctive culture. I don't often interact with the communities that were on the instance, but I always enjoy seeing my peers crop up in the comments of various posts — it's one of my favourite parts of Lemmy being federated (db0 is another example of an instance that has such a distinctive vibe that seeing it as someone's instance is often useful metadata that affects how I parse their comment)
-
you mean an API for bots...
Mobile Apps need an API too.
-
You might have eaten a different cultivar than those exported. In that case, I agree, there are so many different varieties that we never get to taste, supermarkets usually only have 1 or 2 different types
It's baffling to me how customisable some fruit features can be. I know someone who works in plant genomics and cultivation and it's really cool how they can use genomic analysis alongside selective breeding to select for traits. I imagine that imported papayas are cultivars that are easier to transport internationally.
-
After a month and a half downtime all the users will have moved on to other instances. This is essentially a death sentence for the instance and its communities.
Idk, quite a few (me included) made alt accounts for the time being, and we'll return when it becomes possible, especially if we really do migrate to piefed.
-
Mobile Apps need an API too.
why does everything need a mobile app?
-
why does everything need a mobile app?
It's not just native Apps. Alternative web UIs like Thunder, Photon and Voyager need them too.
-
(speaking as a slrpnk user): Another backup communication strategy (once things are backup) might be to designate somewhere on a non-slrpnk instance as a place where people can check for updates if things go down; when I first discovered the outage, I wasn't sure where to go to check for info/updates.
Unrelatedly, I hope that this unexpected outage isn't causing you or other admins too much stress. Whilst the extended nature of this outage is unfortunate, I respect that you're using this as an opportunity to migrate to a more robust solution. This kind of resilience focussed response is a key part of the solarpunk ethos, in my view.
Some people have said that such a long outage seems likely to kill an instance, but for my part, this community is worth waiting for — I have enjoyed having an account on this instance because it feels like the perfect blend of small enough to have a distinct culture and ethos, but is large enough to be robust and diverse.
Thanks.
Yeah at the very least this will result in some changes in the hardware setup to avoid similar situations. But of course it is impossible to forsee all possible problems.
-
I think the migration to Piefed would be worse. Piefed is great, but needs better App support for Lemmy users to migrate. A merge of the Thunder fork into main Thunder would get the ball rolling.
Tag @Kris@feddit.org
On Android you can use the Interstellar app https://interstellar.jwr.one/ with piefed.social! Also iOS: https://testflight.apple.com/join/JXRtHye2
-
It's not just native Apps. Alternative web UIs like Thunder, Photon and Voyager need them too.
yes, but those frontends are typically tied closer to the backend than a public API.
things like CSRF can help block abuse of the back end.
-
yes, but those frontends are typically tied closer to the backend than a public API.
things like CSRF can help block abuse of the back end.
Nope they all use the public API. Even the default Lemmy web client.
-
That sucks.
-
It wouldn’t be meme-worthy were it lossless.
Is this loss dot jaypeg
-
After a month and a half downtime all the users will have moved on to other instances. This is essentially a death sentence for the instance and its communities.
I dont think it will be. I've had two accounts for a while to deal with unexpected issues & will happily return to my slrpnk account once it's back up and running
Plus for the communities, people who were subscribed to them before will likely stay subscribed & once the instance is back up the posts will be in their feeds without an issue.
Being able to accommodate issues like this is one of the major upsides to a decentralized platform.
-
What is so special about piefed? I see a few communities moving there. The interface looks different from the original lemmy interface.
-
Slrpnk.net admin here.
The failure seems to have been in the main firewall, if it had been the server itself we could have easily restored it on another server from the backups on another machine. But as it stands, remote access is entirely cut off.
There usually is another person with hardware access, but they are on summer holidays. This seemed like an acceptable risk at the time...
An off-site backup would have been nice of course, but due to the costs involved in running an Lemmy instance of that size on a rented server, it would have not been a great option either.
I have plans to add a KVM to the main firewall via a secondary connection, but even that might have not helped in this case. I'll know more when I have physical access again.
I've done a lot of SysAdmin and DCOps stuff in the past so, thought I'd give you some plausible suggestions (haven't dug deep into Lemmy DB stuff and DNS/Federation of the stack, so not sure all is practical).
Scenario 1 - Preserve and merge when access is restored
Setup
- Spin up two VMs/VPS (or one that has enough grunt for two Lemmy servers). Call them
robak.slrpnk.net
andslrpnk.net
and point DNS appropriately. - Pull federated content from other instances and place it on robak, set as read-only.
- Sync important comms to (new) slrpnk.net without content.
- Allow users to sign up, vetting as possible (all mods). Keep a list of those that are vetted (call it vetted.list). Inform all users that any non-vetted users will have their content dropped when access is restored.
Merge!
- Once access is restored, ensure that (old) slrpnk.net is set to read-only.
- Schedule a maintenance window (announce more time than you are likely to need).
- During the maintenance window, put (new) slrpnk.net into R/O, or just block external access.
- Query the db on (old) slrpnk.net for all users.
- Subtract the vetted users from vetted.list from the list.
- Drop all records from the resulting list of non-vetted users from (new) slrpnk.net.
- Insert the records from vetted and new users (those without conflicts) into the DB on (old) slrpnk.net.
- Validate that everything is working
- Cut over DNS and spin down the new VMs/VPS.
Scenario 2 - Server is in DC or Admin able to facilitate access
- Get a db dump/backup.
- Spin up temporary slrpnk.net on a VM/VPS.
- Use backup of temporary server to restore data to original, when possible.
- Spin up two VMs/VPS (or one that has enough grunt for two Lemmy servers). Call them
-
It's clearly "slurp nook," that nook where you can go to slurp soup as loudly as you want without being judged.
Makes most sense
-
Don't pull a feddit.de on people, alright?
What exactly happened there? It was the big thing, then I didn't use it for a month or so and then it was gone.
-
Nope they all use the public API. Even the default Lemmy web client.
well that's poor planning and why bots are such a problem.
I know CSRF tokens aren't a silver bullet, but doing nothing to stop them does nothing to stop them.
-
What is so special about piefed? I see a few communities moving there. The interface looks different from the original lemmy interface.
Well, you can add flairs