Category Archives: Strategy and IT (Information Technology)

The Cloud Cost Optimization Void and connecting FinOps and DevOps teams

Keeping operational costs optimized in the Cloud is one of the main pillars for cloud based companies success. This goes along with resilience, agility, and security. There are many resources discussing cost optimization, but they often lack the critical component—the human factor. Your people and processes will determine your financial success much more than the tools you use. Today we’ll focus on the former.

Let’s look at the basic scenario happening at your company as well: The FinOps (Financial Operations) team taking care of cost control and optimization for your cloud operations picks up on a component or a process that requires a change to reduce cost and improve operations. For example, it could be changing your instance types from the existing generation to the latest one. Moving to the new instance type can cut cost in up to 20%.

The FinOps team reaches out to the relevant development (or DevOps) teams with recommendations to change the instance types they are using. This is where the optimization process can break. Sometimes the development team simply does not respond, or they could state they have challenges (technical, time related or other) to address the required changes. There are cases where development teams accept and execute changes in a timely manner, but there is lack of visibility for the change and its impact on cost. Practically, The cost cut disappears into the “void”. Maybe it will be back sometime, probably too late, same as the passengers of the TV Drama “Manifest” of flight 828 have—and it did not end up well (so far at season 3 :-))

During the many journeys I lead with companies, I found the following process valuable in addressing the cost optimization challenges described here.

First, make sure you have continuous monitoring of your application workloads across their environments (development, QA, Staging, Pre-Production, Production). In each environment, your application workload (resources it uses) may change. Tag your application deployments with their change number (revision, version etc.). That way, you can observe the performance and resource use of your workloads across their change numbers and compare them.

Your FinOps team should be able to open a software/configuration change request for the development teams, when they find a workload that requires that change to improve its cost. That change request should enter the development team tasks. The task should have a cost-reduction estimation assigned to it. You can compute the estimated cost reduction according to the workload current cost, and then applying the estimated cost cut.

If you are a developer, you may say, “but my development team’s tasks don’t have any financial figure attached to them—how would I prioritize the cost reduction task versus a marketing or customer ask task?”. My answer is that most of your tasks could be set with a financial gain figure to them. So adding functionality would have a “5% income gain” assigned to it and could be compared to a “cost cut of 10%”. Lacking that, you should still strive to find a way to prioritize cost-cut related tasks.

Once the development team priorities allow it, they develop the code to implement the cost reduction. Since the task is in the development queue everyone knows about it, FinOps team as well- and we have visibility and accountability. Once the code change deploys into an environment (provided it is NOT mixed with other changes) you can clearly observe its effect on cost in your monitoring system: cost of your workload before and after the change.

Of course, this process is still challenging in many cases and having all the components I described can take time and effort to set and maintain, but I believe aiming to it brings valuable benefits.

We are all eager to learn from your experience on that topic, so please share!

Yours,

Jacky Bezalel, Senior Technical Leader at Amazon Web Services ; Teams and Senior Management Career Coach.

AWS Secrets… Yes!!

YES! ! AWS Secrets!!

https://aws.amazon.com/blogs/aws/aws-secrets-manager-store-distribute-and-rotate-credentials-securely/

I’d say secrets in parameter store are like Serverless credentials in Jenkins while secrets in secrets manager are like Serverless hashicorp vault. The difference for now is in the limits of use – SSM is free but would not work well when saturated with many calls – you are expected to use it moderately, while in AWS secrets you are not limited cause you pay. I believe in the future AWS secrets will be more feature-rich.

Redhat Cloud Forms takes a bite into Cloud and Configuration Management

The latest Cloud Forms from Redhat targets the easy use of AWS Cloud Formation and OpenStack Heat templates import, customization, creation, deployment.

It offers a service catalog of Cloud resources setups including load balancers, servers and more.

It also makes it easier to customize your Cloud templates by offering forms and variables per the templates you pick.

Then it triggers Ansible Tower for in depth deployment and configuration management of your instances.

The Cloud Management portal shows you your Cloud components, instances, operating systems and applications including general Linux and Windows as well.

Sounds perfect?

Maybe it sounds like an enterprise vendor trying to grab it all..and maybe this time this vendor actually makes it..

I still would like to see TerraForm there as well..

Anyway there’s the video

Kubernetes- making it Highly Available

You can set a Highly Available Kubernetes cluster  by adding worker node pools and master replicas.

That’s true as of Kubernetes version 1.5.2. It is supported using the kube-up/kube-down scripts for GCE (as alpha): http://blog.kubernetes.io/2017/02/highly-available-kubernetes-clusters.html?m=1

For AWS you have support for HA Kubernetes cluster using KOPS scripts:

http://kubecloud.io/setup-ha-k8s-kops/

GCP Big Table – main facts

GCP Big Table – main facts:

Is the basis of many google products
Object storage system

Does not offer indexes except for a single range index you can use

Is the basis for Hadoop big data system

You pay for storage separately

You pay for min 3 nodes and can expand as you need

Nodes are needed just for read / write – not for storage

Support for massive amounts of reads / writes but not locking or transaction support

Is not completely and highly available since sometimes data is not available as it is moved around

Great for big queries, less for short quick rapid ones

https://cloud.google.com/bigtable/

Why managers are pushed away by Talents and Leaders

Here are the main points in this brilliant session by Ade McCormack on the new age of workplace, employment and skills

Managers were needed in the Industrial Age to keep an eye on employees that did not like the automatic tasks they had to do
Now automation took over those jobs

There is no room for laziness

The need is for talented people and leaders

Those are passionate people who are eager to do much more than any manager could demand

Talents look for innovation, mobility/flexibility/fun-ability , work/life balance, playing with other great people – all those so much worthy than money, yet so hard to create

When you create a great workplace, great people will join, driving great customers to you as they provide massive value

Nowadays risk is hooked to value

You have to bring risk into your work plans to make real progress

As you play in risky arenas you need some peripheral sensory – data about new risks

You need to spend time trying out the new risk related technologies even if there is no guarantee they will materialize into actual danger

Leaders must make sure their teams don’t have any interference to gain and maintain a state of “flow” – joyful focus on exercising their maximum abilities

Treat your career as a lean startup – choose what you are passionate about, have great skills at, and got market demand for

Ask yourself everyday- am I working in a place that allows me to gain my maximum market value

How hard do you practice and learn to become world class in your arena?

Solved: DE-Centralized Puppet Asset Management – factbeat

Using a DE-Centralized (Master-Less) Puppet stack has its benefits for dynamic fast morphing environments.

Yet you’d still love to get all changes made to your environment recorded in a central repo.

Check out factbeat from Elasticsearch community. It’s a beat that  ships Puppet Facter facts to Elasticsearch, where they can be stored, analyzed, displayed and compared over time.

Factor can be easily customized to ship new types of configuration information as your heart desires.

What are you using?