Serverless Bot. Infrastructure with Pulumi

I like to learn something new: new technologies, languages, countries, food, and experiences — all of that is a big part of my life’s joy.

During thinking about my next OKR goal with folks from JetBridge I came up with an idea for how to build something interesting which can help me learn new words and phrases in a more productive way.

The main idea sounds simple: to implement a with a Telegram app bot for learning new words using Anki periodic repetition algorithm. To make it more interesting (and complex 🙃) I decided that it needs to be serverless with AWS lambda and have infrastructure managed by pulumi. This article describes my approach.

The infrastructure contains:

  • API Gateway to handle incoming requests and pass them forward
  • Lambda to process user commands
  • RDS Postgres database instance for storage
Infrastructure

Pulumi allows managing code infrastructure in a convenient manner using any programming language. It’s great when you can define and configure the infrastructure using the same language as the app. In my case, it is Go.

Another benefit that pulumi provides out of the box is using variables directly from created resources. For example, you create an RDS Instance, and then an endpoint and DB properties are available to be used as lambda env variables. It means you don’t need to hard-code such variables or store them anywhere, pulumi manages them for you. Even in the case when some resource is changed (for example a major RDS upgrade will create a new instance) all related infrastructure will be up to date.

The bigger project would require a much more complex definition and resources stack, but for my purpose, it’s rather small and can be kept in only one file.

Resources created by Pulumi

One more cool feature about pulumi is managing secrets. Usually, you need to configure and use some service for it (AWS Secret store, Vault, etc) but with Pulumi you can use it out of the box with a simple command:

pulumi config set --secret DB_PASSWORD <password>

And then use it as an environmental or any other specified property of your app. Additionally, you can configure a key for encryption which is stored in pulumi by default but can easily be changed to a key from AWS KMS or other available provider:

Exported properties allow us to see the most important infra params directly in the CLI or pulumi interface:

ctx.Export("DB Endpoint", pulumi.Sprintf("%s", db.Endpoint))
Exported outputs

To deploy a lambda update, it needs to be built and archived. Then pulumi just sends this archive as a lambda source:

 function, err := lambda.NewFunction(
   ctx,
   "talkToMe",
   &lambda.FunctionArgs{
    Handler: pulumi.String("handler"),
    Role:    role.Arn,
    Runtime: pulumi.String("go1.x"),
    Code:    pulumi.NewFileArchive("./build/handler.zip"),
    Environment: &lambda.FunctionEnvironmentArgs{
     Variables: pulumi.StringMap{
      "TG_TOKEN":    c.RequireSecret("TG_TOKEN"),
      "DB_PASSWORD": c.RequireSecret("DB_PASSWORD"),
      "DB_USER":     db.Username,
      "DB_HOST":     db.Address,
      "DB_NAME":     db.DbName,
     },
    },
   },
   pulumi.DependsOn([]pulumi.Resource{logPolicy, networkPolicy, db}),
  )

To have it as a simple command I use a Makefile:

build::
 GOOS=linux GOARCH=amd64 go build -o ./build/handler ./src/handler.go
 zip -j ./build/handler.zip ./build/handler

Then I can trigger deploy just with make build && pulumi up -y

After that pulumi defines if the archive was changed and sends an update if so. For my simple project, it takes about 20s which is quite fast.

GOOS=linux GOARCH=amd64 go build -o ./build/handler ./src/handler.go
zip -j ./build/handler.zip ./build/handler
updating: handler (deflated 45%)

View Live: https://app.pulumi.com/<pulumi-url>

     Type                    Name                Status            Info
     pulumi:pulumi:Stack     studyAndRepeat-dev
 ~   └─ aws:lambda:Function  talkToMe            updated (12s)     [diff: ~code]


Outputs:
    DB Endpoint   : "<endpoint>.eu-west-1.rds.amazonaws.com:5432"
    Invocation URL: "https://<endpoint>.execute-api.eu-west-1.amazonaws.com/dev/{message}"

Resources:
    ~ 1 updated
    12 unchanged

Duration: 20s

Here you can take a look at the whole infrastructure code.

In conclusion, I can say that pulumi is a great tool to write and keep infrastructure structured, consistent and readable. I used CloudFormation and Terraform previously and pulumi gives me a more user-friendly and pleasant way to have infrastructure as code.

5 tips to make your DevOps team thrive

Written By Vinicius Schettino

DevOps Culture made a real dent on the traditional way organizations used to make software. Heavily inspired by Agile and Lean movements, the DevOps trend makes a move against siloed knowledge, slow and manual deployment pipelines, and strong division between teams such as Development, Testing and Operations. The results announced have been more than eye-catching: Faster deliveries and consequent feedback, happier engineers and better outcomes from the customer/business perspective. 

Background

Aside from the incentives to adopt DevOps practices, companies still struggle to achieve their goals when transforming their internal and customer-facing processes. Turns out that the path is harder than most think. In order to really transform outdated practices into collaborative, agile and automated approaches, the shift cannot be contained only to technical aspects of software development. DevOps Transformation must encompass all organization levels, from developers to executive management. 

Companies often try to achieve that through creating DevOps teams, responsible for the steering and introduction of DevOps practices across the organization. This approach has several pitfalls and threats to efficacy. Often the pushback for maintaining power structures and status quo diminishes the outcomes of that strategy. Instead of really transforming the development process, they often get a limited effect into given phases (such deployment) and automation. When you think of Software Development as a complex system In practice it means that the outcomes are quite limited and the investment might not even pay off. 

In order to avoid that, there are several strategies that can help to avoid said pitfalls and increase the impact of a DevOps team on an organization. In this article we are going to present some tips that can be used to propose, improve and organize DevOps teams in order to make the most out of them. 

1) Make the effort collaborative

The first aspect of DevOps that must be made clear for everyone is that it’s not a thing for only one team to do. The whole idea behind DevOps is about collaboration, avoiding handoffs and complex communication between those working on software. It means it’s impossible to expect one company to “be DevOps” where just one group is trying to put DevOps practices inplace. Those should be widespread, from product management to security and monitoring. Each step towards working software must embrace core concepts of DevOps.

Please note that this is not to say DevOps teams shouldn’t exist. On the contrary, the DevOps world has come a long way, with enough tools, techniques, approaches that we can’t expect everyone to dominate such a variety of skills. The intersection between “Dev” and “Ops” might be big enough to have people dedicated to that. It’s more about what the DevOps team wants to achieve and how they are going to get there. The posture is everything: instead of working alone in the dark and owning everything CI/CD/Ops/Security related, the team must pursue exactly the contrary: working along with product teams, public decisions, pair/mob programming and much more. Might sound simple, but the collaborative approach might be the most important and most ignored aspect of a DevOps Transformation.  In summary, the DevOps team cannot be a silo for DevOps knowledge. Which leads us to the next topic…

2) Do not be the traditional Ops team

Prior to the DevOps era, most organizations would have at least two different teams involved in software production: Development and Operations. While the first would be responsible for product thinking, coding, testing, quality assurance, the second would work on infrastructure, delivery, security, reliability, and monitoring. That way the ideas were transformed into working software through a process resembling an assembly line. Depending on factors such as size, maturity, capillarity, industry sector and even technology, those teams were subdivided into a handful of other, more specialized and shortsighted groups.

Back then, although the frontier between Development and Operations was, naturally, blurred, each team would have quite specific skills and attributions, giving the whole streamed process a frontdesk, bureaucrat tone. 

In that context we had, guess what? A (set of) team(s) dedicated to Operations filled with the same people that today (mostly) form DevOps and similar teams, such as SRE and Engineering Platforms. However, the new proposal has several differences in goals and practices, including the need of working collaboratively in cross functional, small and focused teams. Sadly, far too often they are treated and viewed as the old Operation silo. That way they must “lead by example” the way up into DevOps transformation; not just DevOps tooling and technical aspects, but also through the actions and drive. 

Needless to say, working with assembly-line-like organized teams is very far from an Agile mindset, on which the DevOps principles rely a lot. That said, the DevOps team must be as far as possible from the traditional, waterfall practices of the Operation Teams.

3) Fight the topology that suits your context

There’s no such a thing as perfect architecture! The same could be said for how teams entangled to DevOps should be organized. Every organization is a unique complex system (remember?) and because of that, each solution must be perfectly tailored for the current place it’ll be deployed. That said, however, there are some common patterns that can be found in the wide. Team Topologies identifies the fundamental types of teams and the interactions between them. Following their taxonomy, DevOps focused groups should lean towards the category of Enabling Teams, acting as advocates, evangelists, and overall stimulators of DevOps practices among the stream-aligned teams. Other forms of organization can be of course useful; SRE and Platform Engineering related strategies gravitate towards Platform Teams. What’s the course of action? Just let the software people organize themselves seeking to fulfill the organization/product goals. The resulting team topology will reflect the specific needs of that domain and the support to the initiative will be the strongest possible. That’s a huge success factor since collaboration and will play a big part when the DevOps team starts. 

4) Tools, Templates and Documentation: ways of working together

A key decision when structuring a DevOps team is how they are going to collaborate with others in order to introduce DevOps thinking into software development. Remember DevOps teams should be constantly in touch with other teams presenting new tech, strategies and principles that will be then incorporated (heterogeneously, I would say) throughout the company. The first way one could think of is through traditional ways of collaboration: emails, sync meetings and in person talks appointments.

In spite of those having value, they are far from being really efficient for DevOps collaboration and maybe even for knowledge sharing altogether. There are a few approaches that successful DevOps Teams usually take advantage of:

  • Workshops: Yes, the traditional hands-on course! The idea is old, inspired by classrooms, but the execution changed a lot. Today it can be transformed into a very efficient way of presenting new tools and techniques to a broader public. The emphasis on relevant topics to the public and the use of real world scenarios to train and understand team members are key to the success of workshops.
  • Templates: Sometimes DevOps best practices can be translated into a set of basic tools and configurations to be used on new pieces of documentation and software. In these cases, templates can be a very efficient way of sharing knowledge. It might contain config files for CI/CD platforms, deployment instructions, Dockerfiles, security checks and much more! Just be sure there’s enough flexibility for the use cases you want to cover and enough information so people won’t blindly follow some mysterious, unchangeable standard.
  • Documentation: are you thinking about a boring, gigantic white document with ugly fonts and little to no figures? Think again! Many other forms of storing and sharing knowledge could fall into this category: blog posts, videos, podcasts and even tutorials. The logic is simple: how much do you learn by tutorials and similar content you find online? Why wouldn’t a company create a knowledge source like that? Some content might be too specific and sensitive to be internal, others can also be public and contribute to the increase the awareness of your brand from an employer/reference point of view.  
  • Tools: I think most of the tools related to DevOps topics should be owned by Platform Engineering and SRE teams. It’s another type of collaboration that should be addressed with somewhat different strategies that I intend to write later. 

In all those cases, it’s very important for the DevOps team members to actually understand what their colleagues need. Treat them as your clients and your work as products and ensure what you’re building is really making their lives better. Also, it’s unsurprisingly common to find all those (and others) approaches combined into a greater knowledge sharing strategy.

5) DevOps team scalability

The harsh truth must be said: companies want as much as possible of their software people turning their attention to building value for clients. That pretty much translates as features and working software, no matter how hard it is to keep the big machine working. So we usually cannot afford (and maybe we shouldn’t anyway) to have DevOps people growing at the same pace the rest of the teams grow. That’s ok, we can scale quite efficiently: the ways of collaborating are cheap, easy to maintain and can often be directed to hundreds of professionals at the same time. Onboarding, periodic study groups or tech talks also have a far reach. Another aspect that calls for scabillity is how big is the umbrella on which DevOps things fall. We could add to the Original terms at least a few others 

As teams get more mature on DevOps principles, the support they demand becomes more specialized. That’s when the DevOps team can be transformed into smaller, focused groups containing a certain skillset. This often leads towards Security, Observability, CI/CD and other specialties that will act to teach and show to product teams an in-depth view of the underlying field. As paradoxical as it might sound, DevOps teams wouldn’t be needed in a perfect (hypothetical) DevOps company.

Conclusion

Sure we all can tell the benefits of adopting DevOps principles into software development, but getting there is not easy. Some organizations try to form DevOps teams, expecting they’ll lead the way towards a brighter future. The original idea quite often falls behind, manifesting anti-types that really limit their reach. In this brief article we presented some tips that encompass lots of further reading, seeking the experiment and consolidating several principles that will make your DevOps teams thrive.