Production – The Eden of online products and services. Where everything blossoms. And worlds burn down.
We’ve previously covered pretty much all the other environments: Development, Testing and Staging. So here we are. Basically, everything that you’ve worked on is getting downright serious with the Production Environment. All the magic happens here.
Once you take your toy in production, people will – hopefully – start using it. And from that moment on, you always have to think about them when you alter the way your product works.
Think about your product as a train. Make it a high-speed train for extra drama. You build it and test it how much you want, but when people get on the train, everything changes. You can easily go from utter excitement to crying in a corner. Onboarding customers is a responsibility you’ll have to live with. Of course, it’s not a life sentence. If things go really wrong, you can always and ultimately (and slowly) bring the train to a full stop, thank the customers, give them their money back and maybe a small map to help them figure out what they could do next.
Don’t mess with the data
The production database is a fragile thing. You don’t just alter it without thinking about consequences. If the information needs to be in a new state, you’ll have to migrate it somehow, not just drop the existing data and add some new one.
Working on mechanics while the train is moving
When launching new features, after they have been tested and tested some more during Staging you’ll have to be careful, since users might be using your product at that exact time. Put yourself in their shoes! Typing a long, fluffy description of something and getting a scarry error page when clicking submit is definitely not a way to keep your customers happy.
When you’re having too many users for your current infrastructure, you’ll need to scale it. Scaling, when talking about your server’s hardware, can be done either horizontally or vertically.
Horizontal scaling means adding more machines to a distributed system. For example, distributing your database across multiple machines. Need more database power? Just add some more machines.
Vertical scaling is adding more resources to an existing machine. Like adding more memory or a bigger hard drive. Each of these have tradeoffs and limitations. You can’t just put 256 GB of RAM in your home computer, can you?
Scaling a running service is a tough job. It’s just like changing the wheels of your train while the train is moving at 200 km/h. The good part is that if your service needs scaling, then it has lots of users and people love your product. The bad part is that it requires careful planning and flawless execution. Or your train will blow up in some forgotten forest on the tracks. Along with your bright plans for the future. And your investor’s money. And your reputation. Do I need to go on?
Scaling in “the cloud”
When you hear people talking about hosting their service in “the cloud” it usually means on a virtualized infrastructure, with a hardware infrastructure behind it that they have no idea about. That’s because it doesn’t matter. Virtualization is a vertical kind of scalability, but without its limitations. And most importantly, it involves paying as you go. It’s like starting out with a tiny machine (I just love talking about servers as machines) that has 512 MB RAM and a CPU power of a hamster, and 5 minutes later turning it into a massive 36 CPU cores, 256 GB RAM machine that could tear a hole in the space-time continuum. With just a few clicks. And some more money. Awesome, right? Even more awesome is that you can “automagically” configure the system to dynamically reconfigure its technical specifications based on its own usage and load. It’s like your hamster pet that turns itself into a panther when it gets really hungry and needs to hunt down a deer.
But let’s leave the wild beasts alone and get back to our “sheeps”.
Keep your customers informed
Whether is a new feature, some planned maintenance (trains do have to undergo maintenance from time to time) or just some news that you’d want to share with the world, always keep your customers informed.
Don’t let your customers down
With time, users will start trusting you and your service and will rely on it for them to do their thing. Do not let them down! No, seriously: DO NOT let them down! In good times and in bad, it’s never a good idea to keep your users in the dark. They DO NOT want surprises from you. If you fucked up somewhere, it’s way better to send a “We’re sorry, we really fucked it up this time” email than to cover it up and reply to customer support tickets with “Whoa!!… What do you mean your data is gone?!” And even if this is not a DevOps pure concept, technical problems should not get covered up. Be honest!
Transparency will take you very far in the online work environment.