Redis, best known as a data cache or real-time data platform, is evolving into much more, Tim Hall, chief of product at the company told The New Stack in a recent TNS Makers podcast.

Redis is an in-memory database or memory-first database, which means the data lands there and people are using us for both caching and persistence. However, these days, the company has a number of flexible data models, but one of the brand promises of Redis is developers can store the data as they're working with it. So as opposed to a SQL database where you might have to turn your data structures into columns and tables, you can actually store the data structures that you're working with directly into Redis, Hall said.

Primary Database?

“About 40% of our customers today are using us as a primary database technology,” he said. “That may surprise some people if you're sort of a classic Redis user and you knew us from in-memory caching, you probably didn't realize we added a variety of mechanisms for persistence over the years.”

Meanwhile, to store the data, Redis does store it on disk, sort of behind the scenes while keeping a copy in memory. So if there's any sort of failure, Redis can recover the data off of disk and replay it into memory and get you back up and running. That's a mechanism that has been around about half a decade now.

Yet, Redis is playing what Hall called the ‘long game', particularly in terms of continuing to reach out to developers and showing them what the latest capabilities are.

“If you look at the top 10 databases on the planet, they've all moved into the multimodal category. And Redis is no different from that perspective” Hall said. “So if you look at Oracle it was traditionally a relational database, Mongo is traditionally JSON documents store only, and obviously Redis is a key-value store. We've all moved down the field now. Now, why would we do that? We're all looking to simplify the developer’s world, right?”

Yet, each vendor is really trying to leverage their core differentiation and expand out from there. And the good news for Redis is speed is its core differentiation.

“Why would you want a slow data platform? You don't, Hall said. “So the more that we can offer those extended capabilities for working with things like JSON, or we just launched a data structure called t-digest, that people can use along and we've had support for Bloom filter, which is a probabilistic data structure like all of these things, we kind of expand our footprint, we're saying if you need speed, and reducing latency, and having high interactivity is your goal Redis should be your starting point. If you want some esoteric edge case functionality where you need to manipulate JSON in some very strange way, you probably should go with Mongo. I probably won't support that for a long time. But if you're just working with the basic data structures, you need to be able to query, you need to be able to update your JSON document. Those straightforward use cases we support very, very well, and we support them at speed and scale.”

Customer View

As a Redis customer, Alain Russell, CEO at Blackpepper, a digital e-commerce agency in Auckland, New Zealand, said his firm has undergone the same transition.

“We started off as a Redis as a cache, that helped us speed up traditional data that was slower than we wanted it,” he said. “And then we went down a cloud path a couple of years ago. Part of that migration included us becoming, you know, what's deemed as ‘cloud native.’ And we started using all of these different data stores and data structures and dealing with all of them is actually complicated. You know, and from a developer perspective, it can be a bit painful.”

So, Blackpepper started looking for how to make things simpler, but also keep their platform very fast and they looked at the Redis Stack. “And honestly, it filled all of our needs in one platform. And we're kind of in this path at the moment, we were using the basics of it. And we're very early on in our journey, right? We're still learning how things work and how to use it properly. But we also have a big list of things that we're using other data stores for traditional data, and working out, okay, this will be something that we will migrate to, you know, because we use persistent heavily now, in Redis.”

Twenty-year-old Blackpepper works with predominantly traditional retailers and helps them in their omni-channel journey.

Commercial vs. Open Source

Hall said there are three modes of access to the Redis technology: the Redis open source project, the Redis Stack – which the company recommends that developers start with today -- and then there's Redis Enterprise Edition, which is available as software or in the cloud.

“It's the most popular NoSQL database on the planet six years running,” Hall said. “And people love it because of its simplicity.”

Meanwhile, it takes effort to maintain both the commercial product and the open source effort. Allen, who has worked at Hortonworks, InfluxData, said “Not every open source company is the same in terms of how you make decisions about what lands in your commercial offering and what lands in open source and where the contributions come from and who's involved.”

For instance, “if there was something that somebody wanted to contribute that was going to go against our commercial interest, we probably not would not merge that,” Hall said.

Redis was run by project founder Salvatore Sanfilippo, for many, many years, and he was the sole arbiter of what landed and what did not land in Redis itself. Then, over the last couple of years, Redis created a core steering committee. It's made up of one individual from AWS, one individual from Alibaba, and three Redis employees who look after the contributions that are coming in from the Redis open source community members who want to contribute those things.

“And then we reconcile what we want from a commercial interest perspective, either upstream, or things that, frankly, may have been commoditized and that we want to push downstream into the open source offering, Hall said. “And so the thing that you're asking about is sort of my core existential challenge all the time, that is figuring out where we're going from a commercial perspective. What do we want to land there first? And how can we create a conveyor belt of commercial opportunity that keeps us in business as a software company, creating differentiation against potential competitors show up? And then over time, making sure that those things that do become commoditized, or maybe are not as differentiating anymore, I want to release those to the open source community. But this upstream/downstream kind of challenge is something that we're constantly working through.”

Blackpepper was an open source Redis user initially, but they started a journey where they used Memcached to speed up data. Then they migrated to Redis when they moved to the AWS cloud, Russell said.

Listen to the Podcast

The Redis TNS Makers podcast goes on to look at the use of AI/ML in the platform, the acquisition of RESP.app, the importance of JSON and RediSearch, and where Redis is headed in the future.

Back in April, Kris Nóva, now principal engineer at GitHub, started creating a server on Mastodon as a side project in her basement lab.

Then in late October, Elon Musk bought Twitter for an eye-watering $44 billion, and began cutting thousands of jobs at the social media giant and making changes that alienated longtime users.

And over the next few weeks, usage of Nóva’s hobby site, Hachyderm.io, exploded.

“The server started very small,” she said on this episode of The New Stack Makers podcast. “And I think like, one of my friends turned into two of my friends turned into 10 of my friends turned into 20 colleagues, and it just so happens, a lot of them were big names in the tech industry. And now all of a sudden, I have 30,000 people I have to babysit.”

Though the rate at which new users are joining Hachyderm has slowed down in recent days, Nóva said, it stood at more than 38,000 users as of Dec. 20.

Hachyderm.io is still run by a handful of volunteers, who also handle content moderation. Nóva is now seeking nonprofit status for it with the U.S. Internal Revenue Service, with intentions of building a new organization around Hachyderm.

This episode of Makers, hosted by Heather Joslyn, TNS features editor, recounts Hachyderm’s origins and the challenges involved in scaling it as Twitter users from the tech community gravitated to it.

Nóva and Joslyn were joined by Gabe Monroy, chief product officer at DigitalOcean, which has helped Hachyderm cope with the technical demands of its growth spurt.

HugOps and Solving Storage Issues

Suddenly having a social media network to “babysit” brings numerous challenges, including the technical issues involved in a rapid scale up. Monroy and Nóva worked on Kubernetes projects when both were employed at Microsoft, “so we’re all about that horizontal distribution life.” But the Mastodon application’s structure proved confounding.

“Here I am operating a Ruby on Rails monolith that's designed to be vertically scaled on a single piece of hardware,” Nóva said. “And we're trying to break that apart and run that horizontally across the rack behind me. So we got into a lot of trouble very early on by just taking the service itself and starting to decompose it into microservices.”

Storage also rapidly became an issue. “We had some non-enterprise but consumer-grade SSDs. And we were doing on the order of millions of reads and writes per day, just keeping the Postgres database online. And that was causing cascading failures and cascading outages across our distributed footprint, just because our Postgres service couldn't keep up.”

DigitalOcean helped with the storage issues; the site now uses a data center in Germany, whose servers DigitalOcean manages. (Previously, its servers had been living in Nóva’s basement lab.)

Monroy, longtime friends with Nóva, was an early Hachyderm user and reached out when he noticed problems on the site, such as when he had difficulty posting videos and noticed other people complaining about similar problems.

“This is a ‘success failure’ in the making here, the scale of this is sort of overwhelming,” Monroy said. “So I just texted Nóva, ‘Hey, what's going on? Anything I could do to help?’

“In the community, we like to talk about the concept of HugOps, right? When people are having issues on this stuff, you reach out, try and help. You give a hug. And so, that was all I did. Nóva is very crisp and clear: This is what I got going on. These are the issues. These are the areas where you could help.”

Sustaining ‘the NPR of Social Media’

One challenge in particular has nudged Nóva to seek nonprofit status: operating costs.

“Right now, I'm able to just kind of like eat the cost myself,” she said. “I operate a Twitch stream, and we're taking the proceeds of that and putting it towards operating service.” But that, she acknowledges, won’t be sustainable as Hachyderm grows.

“The whole goal of it, as far as I'm concerned, is to keep it as sustainable as possible,” Nóva said. “So that we're not having to offset the operating costs with ads or marketing or product marketing. We can just try to keep it as neutral and, frankly, boring as possible — the NPR of social media, if you could imagine such a thing.”

Check out the full episode for more details on how Hachyderm is scaling and plans for its future, and Nóva and Monroy’s thoughts about the status of Twitter.

Feedback? Find me at @hajoslyn on Hachyderm.io.

Idit Levine’s tech journey originated in an unexpected place: a basketball court. As a seventh grader in Israel, playing in hoops  tournaments definitely sparked her competitive side.

“I was basically going to compete with all my international friends for two minutes without parents, without anything,” Levine said. “I think it made me who I am today. It’s really giving you a lot of confidence to teach you how to handle situations … stay calm and still focus.”

Developing that calm and focus proved an asset during Levine’s subsequent career in professional basketball in Israel, and when she later started her own company. In this episode of The Tech Founder Odyssey podcast series, Levine, founder and CEO of Solo.io, an application networking company with a $1 billion valuation, shared her startup story.

The conversation was co-hosted by Colleen Coll and Heather Joslyn of The New Stack

After finishing school and service in the Israeli Army, Levine was still unsure of what she wanted to do. She noticed her brother and sister’s fascination with computers. Soon enough, she recalled,  “I picked up a book to teach myself how to program.”

It was only a matter of time before she found her true love: the cloud native ecosystem. “It's so dynamic, there's always something new coming. So it's not boring, right? You can assess it, and it's very innovative.”

Moving from one startup company to the next, then on to bigger companies including Dell EMC where she was chief technology officer of the cloud management division, Levine was happy seeking experiences that challenged her technically. “And at one point, I said to myself, maybe I should stop looking and create one.”

Learning How to Pitch

Winning support for Solo.io demanded that the former hoops player acquire an unfamiliar skill: how to pitch. Levine’s company started in her current home of Boston, and she found raising money in that environment more of a challenge than it would be in, say, Silicon Valley.

It was difficult to get an introduction without a connection, she said:  “I didn't understand what pitches even were but I learned how … to tell the story. That helped out a lot.”

Founding Solo.io was not about coming up with an idea to solve a problem at first. “The main thing at Solo.io, and I think this is the biggest point, is that it's a place for amazing technologists, to deal with technology, and, beyond the top of innovation, figure out how to change the world, honestly,” said Levine.

Even when the focus is software, she believes it’s eventually always about people. “You need to understand what's driving them and make sure that they're there, they are happy. And this is true in your own company. But this is also [true] in the ecosystem in general.”

Levine credits the company’s success with its ability to establish amazing relationships with customers – Solo.io has a renewal rate of 98.9% – using a very different customer engagement model that is similar to users in the open source community. “We’re working together to build the product.”

Throughout her journey, she has carried the idea of a team: in her early beginnings in basketball, in how she established a “no politics” office culture, and even in the way she involves her family with Solo.io.

As for the ever-elusive work/life balance, Levine called herself a workaholic, but suggested that her journey has prepared her for it:  “I trained really well. Chaos is a part of my personal life.”

She elaborated, “I think that one way to do this is to basically bring the company to [my] personal life.  My family was really involved from the beginning and my daughter chose the logos. They’re all very knowledgeable and part of it.”

OpenTelemetry project offers vendor-neutral integration points that help organizations obtain the raw materials — the "telemetry" — that fuel modern observability tools, and with minimal effort at integration time. But what does OpenTelemetry mean for those who use their favorite observability tools but don’t exactly understand how it can help them? How might OpenTelemetry be relevant to the folks who are new to Kuberentes (the majority of KubeCon attendees during the past years) and those who are just getting started with observability? 

Austin Parker, head of developer relations, Lightstep and Morgan McLean, director of product management, Splunk, discuss during this podcast at KubeCon + CloudNativeCon 2022 how the OpenTelemetry project has created demo services to help cloud native community members better understand cloud native development practices and test out OpenTelemetry, as well as Kubernetes, observability software, etc. 

At this conjecture in DevOps history, there has been considerable hype around observability for developers and operations teams, and more recently, much attention has been given to helping combine the different observability solutions out there in use through a single interface, and to that end, OpenTelemetry has emerged as a key standard. 

DevOps teams today need OpenTelemetry since they typically work with a lot of different data sources for observability processes, Parker said. “If you want observability, you need to transform and send that data out to any number of open source or commercial solutions and you need a lingua franca to to be consistent. Every time I have a host, or an IP address, or any kind of metadata, consistency is key and that's what OpenTelemetry provides.”

Additionally, as a developer or an operator, OpenTelemetry serves to instrument your system for observability, McLean said. “OpenTelemetry does that through the power of the community working together to define those standards and to provide the components needed to extract that data among hundreds of thousands of different combinations of software and hardware and infrastructure that people are using,” McLean said.

Observability and OpenTelemetry, while conceptually straightforward, do require a learning curve to use. To that end, the OpenTelemetry project has released a demo to help. It is intended to both better understand cloud native development practices and to test out OpenTelemetry, as well as Kubernetes, observability software, etc.,the project’s creators say.

OpenTelemetry Demo v1.0 general release is available on GitHub and on the OpenTelemetry site. The demo helps with learning how to add instrumentation to an application to gather metrics, logs and traces for observability. There is heavy instruction for open source projects like Prometheus for Kubernetes and Jaeger for distributed tracing. How to acquaint yourself with tools such as Grafana to create dashboards are shown. The demo also extends to scenarios in which failures are created and OpenTelemetry data is used for troubleshooting and remediation. The demo was designed for the beginner or the intermediate level user, and can be set up to run on Docker or Kubernetes in about five minutes. 

“The demo is a great way for people to get started,” Parker said. “We've also seen a lot of great uptake from our commercial partners as well who have said ‘we'll use this to demo our platform.’”

Nathan Peck, a senior developer advocate for generative AI at Amazon Web Services (AWS), shares his experiences working with Python in a recent episode of The New Stack Makers, recorded at PyCon US. Although not a Python expert, Peck frequently deals with Python scripts in his role, often assisting colleagues in running scripts as cron jobs. He highlights the challenge of being a T-shaped developer, possessing broad knowledge across multiple languages and frameworks but deep expertise in only a few.

Peck introduces Amazon Q, a generative AI coding assistant launched by AWS in November, and demonstrates its capabilities. The assistant can be integrated into an integrated development environment (IDE) like VS Code. It assists in explaining, refactoring, fixing, and even developing new features for Python codebases. Peck emphasizes Amazon Q's ability to surface best practices from extensive AWS documentation, making it easier for developers to navigate and apply.

Amazon Q Developer is available for free to users with an AWS Builder ID, without requiring an AWS cloud account. Peck's demo showcases how this tool can simplify and enhance the coding experience, especially for those handling complex or unfamiliar codebases.

Learn more from The New Stack about Amazon Q and Amazon’s Generative AI strategy:

Amazon Q, a GenAI to Understand AWS (and Your Business Docs)

Decoding Amazon’s Generative AI Strategy

Responsible AI at Amazon Web Services: Q&A with Diya Wynn

Join our community of newsletter subscribers to stay on top of the news and at the top of your game. 

A major part of improving developer velocity is about getting the most out of an observability platform. While that is a commonly held assumption, this best practice is also a far-reaching goal for many DevOps teams.

Hosted by Alex Williams, founder and publisher of The New Stack, this The New Stack Makers — podcast recorded during a virtual pancake breakfast — features a discussion on improving observability for developers. The featured guests were Zain Asgar, general manager, Pixie and New Relic open source and CEO and co-founder of Pixie Labs, Roopak Venkatakrishnan, engineering manager, Bolt (an e-commerce retailer tool), Ihor Dvoretskyi, developer advocate, Cloud Native Computing Foundation (CNCF) and Christine Wang, senior solutions engineer, Grafana Labs.

Today’s developer seems to be working with more tools than ever. Building a Node.js-based JavaScript application could require over a dozen tools at times to get code out into production. It's easy to get sucked down a rabbit hole and not stay focused. Debugging an application once in production can also be a challenge: You want as much context at your fingertips as needed while maintaining a reasonable signal-to-noise ratio.

Dan O’Brien, a software engineer for feature management platform provider LaunchDarkly, has a personal interest in how to keep from being distracted/staying in the flow when working on a new feature or any piece of code.

In this very latest episode of The New Stack Makers podcast, we ask O'Brien about the complexities he sees in today’s developer workflow, as well as some tips he has to stay “in the zone” when writing code. We’ll also discuss the tools that LaunchDarkly has that can help expedite application development. TNS founder and Publisher Alex Williams, along with TNS Managing Editor Joab Jackson, hosted this podcast.

The Linux Foundation's Open Source Security Foundation (OSSF) is addressing the challenge of timely software component updates to prevent security vulnerabilities like Log4J. In an interview with Alex Williams of The New Stack at the Open Source Summit in Vancouver, Omkhar Arasaratnam, the new general manager of OSSF, and Brian Behlendorf, CTO of OSSF, discuss the importance of making software secure from the start and the need for rapid response when vulnerabilities occur. 

In this conversation, they highlight the significance of Software Bill of Materials (SBOMs), which provide a complete list of software components and supply chain relationships. SBOMs offer data that can aid decision-making and enable reputation tracking of repositories. The interview also touches on the issues with package managers and the quantification of software vulnerability risks. Overall, the goal is to improve the efficiency and effectiveness of software component updates and leverage data to enhance security in enterprise and production environments.

Learn more from The New Stack:

Creating a 'Minimum Elements' SBOM Document in 5 Minutes

Enhance Your SBOM Success with SLSA

First released in 2016, the Svelte Web framework has steadily gained popularity as an alternative approach to building Web applications, one that prides itself on being more intuitive (and less verbose) than the current framework du jour, Facebook's React. You can say that it reaches back to the era before the web app — when desktop and server applications were compiled — to make the web app easier to develop and more enjoyable to user.

In this latest episode of The New Stack Makers podcast, we interview the creator of Svelte himself, Rich Harris. Harris started out not as a web developer, but as a journalist who created the framework to do immersive web journalism. So we were interested in that.

In addition to delving into history, we also discussed the current landscape of Web frameworks, the Web's Document Object Model, the way React.js updates variables, the value of TypeScript, and the importance SvelteKit. We also chatted about why Vercel, where Harris now works maintaining Svelte, wants to make a home for Svelte.

TNS Editor Joab Jackson hosted this conversation.

Below are a few excerpts from our conversation, edited for brevity and clarity.

So set the stage for us. What was the point that inspired you to create Svelte?

To fully tell the story, we need to go way back into the mists of time, back to when I started programming. My background is in journalism. And about a decade ago, I was working in a newsroom at a financial publication in London. I was very inspired by some of the interactive journalism that was being produced at places like the New York Times, but also the BBC and the Guardian and lots of other news organizations, where they were using Flash and increasingly JavaScript, to tell these data rich interactive stories that couldn't really be done any other way.

And to me, this felt like the future of journalism, it's something that was using the full power of the web platform as a storytelling medium in a way that just hadn't been done before. And I was very excited about all that, and I wanted a piece of it.

So I started learning JavaScript with the help of the help of some some friends, and discovered that it's really difficult. Particularly if you're doing things that have a lot of interactivity. If you're managing lots of state that can be updated in lots of different ways, you end up writing what is often referred to as spaghetti code.

And so I started building a toolkit, really, for myself. And this was a project called Reactive, short for interactive, something out of a out of a Neal Stephenson book, in fact, and it actually got a little bit of traction, not it was never huge, but you know, it was my first foray into open source, and it got used in a few different places.

And I maintained that for some years, and eventually, I left that company and joined the Guardian in the U.K. And we used Reactive to build interactive pieces of journalism there, I transferred to the U.S. to continue at the guardian in New York. And we use directive quite heavily there as well. After a while, though, it became apparent that, you know, as with many frameworks of that era, it had certain flaws.

A lot of these frameworks were built for an era in which desktop computing was prevalent. And we were now in firmly in this age of mobile, first, web development. And these frameworks weren't really up to the task, primarily because they were just too big, they were too big, and they were too bulky and they were too slow.

And so in 2016, I started working on what was essentially a successor to that project. And we chose the name Svelte because it has all the right connotations. It's elegant, it's sophisticated. And the idea was to basically provide the same kind of development experience that people were used to, but change the was that translated into the experience end users have when they run it in the browser.

It did this by adopting techniques from the compiler world. The code that you write doesn't need to be the code that actually runs in the browser. Svelte was really one of the first frameworks to lean into the compiler paradigm. And as a result, we were able to do things with much less JavaScript, and in a way that was much more performant, which is very important if you're producing these kinds of interactive stories that typically involve like a lot of data, a lot of animation

Can you talk a bit about more about the compiler aspect? How does that work with a web application or web page?

So, you know, browsers run JavaScript. And like nowadays, they can run WASM, too. But JavaScript is the language that you need to write stuff in if you want to have interactivity on a web page. But that doesn't mean that you need to write JavaScript, if you can design a language that allows you to describe user interfaces in a more natural way, then the compiler could turn that intention into the code that actually runs. And so you get all the benefits of declarative programming but without the drawbacks that historically have accompanied that.

There is this trade off that historically existed: the developer wants to write this nice, state driven declarative code and the user doesn't want to have to wait for this bulky JavaScript framework to load over the wire. And then to do all of this extra work to translate your declarative intentions into what actually happens within the browser. And the compiler approach basically allows you to, to square that circle, it means that you get the best of both worlds you're maximizing the developer experience without compromising on developer experience.

Stupid question: As a developer, if I'm writing JavaScript code, at least initially, how do I compile it?

So pretty much every web app has a build step. It is possible to write web applications that do not involve a build step, you can just write JavaScript, and you can write HTML, and you can import the JavaScript into the HTML and you've got a web app. But that approach, it really doesn't scale, much as some people will try and convince you otherwise.

At some point, you're going to have to have a build step so that you can use libraries that you've installed from NPM, so that you can use things like TypeScript to optimize your JavaScript. And so Svelte fits into your existing build step. And so if you have your components that are written in Svelte files, it's literally a .SVELTE extension. Then during the build step, those components will get transformed into JavaScript files.

Svelte seemed to take off right around the time we heard complaints about Angular.js. Did the frustrations around Angular help the adoption of Svelte?

Svelte hasn't been a replacement for Angular because Angular is a full featured framework. It wants to own the entirety of your web application, whereas Svelte is really just a component framework.

So on the spectrum, you have things that are very focused on individual components like React and Vue.js and Svelte. And then at the other end of the spectrum, you have frameworks like Angular, and Ember. And historically, you had to do the work of taking your component framework and figuring out how to build the rest of the application unless you were using one of these full-featured frameworks.

Nowadays, that's less true because we have things like Next.js, and remix-vue, And on the Svelte team are currently working on SvelteKit, which is the answer to that question of how do I actually build an app with this?

I would attribute the growth in popularity is felt to different forces. Essentially, what happened is it trundled along with a small but dedicated user base for a few years. And then in 2019, we released version three of the framework, which really rethought the authoring experience, the syntax that you use to write components and, and the APIs that are available.

Around that time, I gave a couple of conference talks around it. And that's when it really started to pick up steam. Now, of course, we're growing very rapidly. And we're consistently at the top of developer-happiness surveys. And so now, like a lot of people are aware of is, but we're still like a very tiny framework, compared to the big dogs like React and Vue.

You have said that part of the Svelte mission has been to make web development fun. What are some of Svelte's attributes that make it less aggravating for the developer?

The first thing is that you can write a lot less code. If you're using Svelte, then you can express the same concepts with typically about 40% less code. There's just a lot less ceremony, a lot less boilerplate.

We're not constrained by JavaScript. For example, the way that you use state inside a component with React, you have to use hooks. And there's this slightly idiosyncratic way of declaring a local piece of state inside the component. With Svelte, you just declare a variable. And if you assign a new value to that variable, or if it's an object, and you mutate that object, then the compiler interprets that as a sign that it needs to update the component.

Telecoms are not necessarily associated with adopting new-generation technologies. However, Deutsche Telekom has made considerable investments cloud in native environments, by creating and supporting Kubernetes clusters to supports its operations infrastructure. 

In this episode of The New Stack Makers podcast, recorded on the floor of KubeCon + CloudNativeCon Europe 2022, DevOps engineers Christopher Dziomba and Samy Nitsche of Deutsche Telekom discuss how one of Europe’s largest telecom providers made the shift to cloud native.

Deutsche Telekom obviously didn’t start from scratch. It had decades worth of telecom infrastructure and networks that all needed to be integrated into the new world of Kubenetes. This involved a lot of “discussion with the other teams,” Dziomba said. “We had to work together [with other departments] to see how we wanted to manage legacy integration, and especially, and especially, policy and process integration,” Dziomba said. 

As it turned out, many of the existing services Deutsche Telekom offered were conductive to integrating into the distributed Kubernetes infrastructure. “It was suited to be deployed on something like Kubernetes,” Dziomba said. “The decision was also made to build the Kubernetes platform by ourselves inside Deutsche Telekom and not to buy one. This really  facilitated the move towards cloud native infrastructure.”

The shift also heavily involved the vendors that were “coming from the old route,”  Nitsche said. “It's sometimes a challenge to make sure that the application is really also cloud native and to make sure it can use all the benefits Kubernetes offers.

Mike Fiedler, a PyPI safety and security engineer at the Python Software Foundation, prefers the title “code gardener,” reflecting his role in maintaining and securing open source projects. Recorded at PyCon US, Fiedler explains his task of “pulling the weeds” in code—handling unglamorous but crucial aspects of open source contributions. Since August, funded by Amazon Web Services, Fiedler has focused on enhancing the security of the Python Package Index (PyPI). His efforts include ensuring that both packages and the pipeline are secure, emphasizing the importance of vetting third-party modules before deployment.

One of Fiedler’s significant initiatives was enforcing mandatory two-factor authentication (2FA) for all PyPI user accounts by January 1, following a community awareness campaign. This transition was smooth, thanks to proactive outreach. Additionally, the foundation collaborates with security researchers and the public to report and address malicious packages.

In late 2023, a security audit by Trail of Bits, funded by the Open Technology Fund, identified and quickly resolved medium-sized vulnerabilities, increasing PyPI's overall security. More details on Fiedler's work are available in the full interview video.

Learn more from The New Stack about PyPl:

PyPl Strives to Pull Itself Out of Trouble

How Python Is Evolving

Poisoned Lolip0p PyPI Packages

Join our community of newsletter subscribers to stay on top of the news and at the top of your game. 

In this episode, Stefano Maffulli, Executive Director of the Open Source Initiative, discusses the need for a new definition as AI differs significantly from open source software. The complexity arises from the unique nature of AI, particularly large language models and transformers, which challenge traditional copyright frameworks. Maffulli emphasizes the urgency of establishing a definition for open source AI and discusses an ongoing effort to release a set of principles by the year's end.

The concept of "open" in the context of AI is undergoing a significant transformation, reminiscent of the early days of open source. The recent upheaval at OpenAI, resulting in the removal of CEO Sam Altman, reflects a profound shift in the technology community, prompting a reconsideration of the definition of "open" in the realm of AI.

The conversation highlights the parallels between the current AI debate and the early days of software development, emphasizing the necessity for a cohesive approach to navigate the evolving landscape. Altman's ousting underscores a clash of belief systems within OpenAI, with a "safetyist" community advocating caution and transparency, while Altman leans towards experimentation. The historical significance of open source, with a focus on trust preservation over technical superiority, serves as a guide for defining "open" and "AI" in a rapidly changing environment.

Learn more from The New Stack about AI and Open Source:

Artificial Intelligence News, Analysis, and Resources

Open Source Development Threatened in Europe

The AI Engineer Foundation: Open Source for the Future of AI

Security-as-code is the practice of “building security into DevOps tools and workflows by mapping out how changes to code and infrastructure are made and finding places to add security checks, tests, and gates without introducing unnecessary costs or delays,” according to tech publisher O’Reilly. In this latest “pancakes and podcast” special episode —recorded during a pancake breakfast during KubeCon + CloudNativeCon in October — we discuss how security-as-code can benefit emerging GitOps practices.

The guests were  Sean O’Dell, director of developer advocacy, Accurics, Sara Joshi, who was an associate software engineer for Accurics when this recording was made;  Parminder Singh, chief information security officer (CISO), for hybrid-cloud digital-transformation services provider DigitalOnUs; Brendan O’Leary, staff developer evangelist, GitLab; Cindy Blake, senior security evangelist, GitLab; and Emily Omier, contributor, The New Stack and owner of marketing consulting provider Emily Omier Consulting.

Alex Williams, founder and publisher of TNS, hosted the podcast.

Deliveroo, a prominent food delivery company, relies on Apache Flink, a distributed processing engine, to enhance its three-sided marketplace, connecting delivery drivers, restaurants, and customers. Seeking to improve real-time data streaming and gain insights into customer behavior, Deliveroo transitioned to Flink, comparing it to alternatives like Apache Spark and Kafka Streams. Flink, with feature parity to their previous platform, offered stability and scalability. They initially experimented with Flink on Kubernetes but turned to the Amazon Managed Service for Flink (MSF) for enhanced support and maintenance.

Engineers from Deliveroo, Felix Angell and Duc Anh Khu, emphasized the need for flexibility in data modeling to accommodate their fast-paced product development. However, flexibility can be complex, often requiring data model adjustments. They expressed the desire for a self-serve configuration feature in MSF, allowing easy customization of low-level settings and auto-scaling based on application metrics. This move to Flink and MSF has empowered Deliveroo to focus on core responsibilities like continuous integration and delivery while efficiently managing their data processing needs.

Learn more from The New Stack about Apache Flink and AWS:

Kinesis, Kafka and Amazon Managed Service for Apache Flink

Apache Flink for Real Time Data Analysis

Apache Flink for Unbounded Data Streams

DETROIT — Even in the midst of hand-wringing at KubeCon + CloudNativeCon North America about how the global economy will make it tough for startups to gain support in the near future, the news about a couple of young WebAssembly-centric companies was bright.

Cosmonic announced that it had raised $8.5 million in a seed round led by Vertex Ventures. And Fermyon Technologies unveiled both funding and product news: a $20 million A Series led by Insight Partners (which also owns The New Stack) and the launch of Fermyon Cloud, a hosted platform for running WebAssembly (Wasm) microservices. Both Cosmonic and Fermyon were founded in 2021.

“A lot of people think that Wasm is this maybe up and coming thing, or it's just totally new thing that's out there in the future,” noted Bailey Hayes, a director at Cosmonic, in this episode of The New Stack Makers podcast.

But the future is already here, she said: “It's one of technology's best kept secrets, because you're using it today, all over. And many of the applications that we use day-to-day —  Zoom, Google Meet, Prime Video, I mean, it really is everywhere. The thing that's going to change for developers is that this will be their compilation target in their build file.”

In this On the Road episode of Makers, recorded at KubeCon here in the Motor City, Hayes and Kate Goldenring, a software engineer at Fermyon, spoke to Heather Joslyn, TNS’ features editor, about the state of WebAssembly. This episode was sponsored by the Cloud Native Computing Foundation (CNCF).

Wasm and Docker, Java, Python

WebAssembly – the roughly five-year-old binary instruction format for a stack-based virtual machine, is designed to execute binary code on the web, lets developers bring the performance of languages like C, C++, and Rust to the web development area.

At Wasm Day, a co-located event that preceded KubeCon, support for a number of other languages — including Java, .Net, Python and PHP — was announced. At the same event, Docker also revealed that it has added Wasm as a runtime that developers can target; that feature is now in beta.

Such steps move WebAssembly closer to fulfilling its promise to devs that they can “build once, run anywhere.”

“With Wasm, developers shouldn't need to know necessarily that it's their compilation target,” said Hayes. But, she added, “what you do know is that you're now able to move that Wasm module anywhere in any cloud. The same one that you built on your desktop that might be on Windows can go and run on an ARM Linux server.”

Goldenring pointed to the findings of the CNCF’s “mini survey” of WebAssembly users,  released at Wasm Day, as evidence that the technology’s user cases are proliferating quickly.

“Even though WebAssembly was made for the web, the number one response —it was around a little over 60% — said serverless,” she noted. “And then it said, the edge and then it said web development, and then it said IoT, and the use cases just keep going. And that's because it is this incredibly powerful, portable target that you can put in all these different use cases. It's secure, it has instant startup time.”

Worlds and Warg Craft

The podcast guests talked about recent efforts to make it easier to use Wasm, share code and reuse it, including the development of the component model, which proponents hope will simplify how WebAssembly works outside the browser. Goldenring and Hayes discussed efforts now under construction, including “worlds” files and Warg, a package registry for WebAssembly. (Hayes co-presented at Wasm Day on the work being done on WebAssembly package management, including Warg.)

A world file, Hayes said, is a way of defining your environment.  "One way to think of it is like .profile, but for Wasm, for a component. And so it tells me what types of capabilities I need for my web module to run successfully in the runtime and can read that and give me the right stuff.”

And as for Warg, Hayes said: “It's really a protocol and a set of APIs, so that we can slot it into existing ecosystems. A lot of people think of it as us trying to pave over existing technologies. And that's really not the case. The purpose of Warg is to be able to slot right in, so that you continue working in your current developer environment and experience and using the packages that you're used to. But get all of the advantages of the component model, which is this new specification we've been working on" at the W3C's WebAssembly Working Group.

Goldenring added another finding from the CNCF survey: “Around 30% of people wanted better code reuse. That's a sign of a more mature ecosystem. So having something like Warg is going to help everyone who's involved in the server side of the WebAssembly space.”

Listen to the full conversation to learn more about WebAssembly and how these two companies are tackling its challenges for developers.

How Kubernetes environments might be able to offer hooks for storage, databases and other sources of persistent data still is a question in the minds of many potential users. To that end, a new consortium called the Data on Kubernetes Community (DoKC)  was formed to help organizations find the best ways of working with stateful data on Kubernetes.

In this latest episode of The New Stack Maker podcast, two members of the group discuss the challenges associated with running stateful workloads on Kubernetes and how DoKC can help.

Participants for this conversation were Melissa Logan, principal, of Constantia.io, an open source and enterprise tech marketing firm, and director of  DoKC; Patrick McFadin, vice president, developer relations and chief evangelist for the Apache Cassandra NoSQL database platform from DataStax; and Evan Powell, advisor, investor and board member, MayaData, a Kubernetes-environment storage-solution provider.

TNS Editor Joab Jackson hosted the podcast.

Proper tooling is perhaps the primary key to unlocking developer productivity. With the right tools and frameworks, developers can be productive in minutes versus having to toil over boilerplate code. And as data-hungry use cases such as AI and machine learning emerge, data tooling is becoming paramount.

This was evident at the recent MongoDB World conference in New York City where TNS Founder and Publisher Alex Williams recorded this episode of The New Stack Makers podcast featuring Peggy Rayzis, senior director of developer experience at Apollo GraphQL; Lee Robinson, vice president of developer experience at Vercel; Ian Massingham, vice president of developer relations and community at MongoDB; and Søren Bramer Schmidt, co-founder and CEO of Prisma, discussing how their companies’ offerings help unlock developer productivity.

Apollo GraphQL and Supergraphs

Apollo GraphQL unlocks developers by helping them build supergraphs, Raysiz said. A supergraph is a unified network of a company's data services and capabilities that is accessible via a consistent and discoverable place that any developer can access with a GraphQL query. GraphQL is a query language for communicating about data.

“And what's really great about the supergraph is even though it's unified, it's very modular and incrementally adoptable. So you don't have to like rewrite all of your backend system and API's,” she said. “What's really great about the Super graph is you can connect like your legacy infrastructure, like your relational databases, and connect that to a more modern stack, like MongoDB Atlas, for example, or even connected to a mainframe as we've seen with some of our customers. And it brings that together in one place that can evolve over time. And we found that it just makes developers so much more productive, helps them shave, shave months off of their development time and create experiences that were impossible before.”

[sponsor_note slug="mongodb" ][/sponsor_note]

Vercel: Strong Defaults

Meanwhile, Robinson touted the virtues of Next.js, Vercel’s popular React-based framework, which provides developers with the tools and the production defaults to make a fast web experience. The goal is to enable frontend developers to be able to move from an idea to a global application in seconds.

Robinson said he believes it’s important for a tool or framework to have good, strong defaults, but to also be extensible and available for developers to make changes such that they do not have necessarily eject fully out of the tool that they're using, but to be able to customize without having to leave the framework library tool of choice.

“If you can provide that great experience for the 90% use case by default, but still allow maybe the extra 10% power, you know, power developer who needs to modify something without having to just rewrite from scratch, you can get go pretty far,” he said.

Data Tooling

When it comes to data tooling, MongoDB is trying to help developers manipulate and work with data in a more productive and effective way, Massingham said.

One of the ways MongoDB does this is through the provision of first-party drivers, he said. The company offers 12 different programming language drivers for MongoDB, covering everything from Rust to Java, JavaScript, Python, etc.

“So, as a developer, you’re importing a library into your environment,” Massingham said. “And then rather than having to construct convoluted SQL statements -- essentially learning another language to interact with the data in your database or data store -- you're going to manipulate data idiomatically using objects or whatever other constructs that are normal within the programming language that you're using. It just makes it way simpler for developers to interact with the data that's stored in MongoDB versus interacting with data in a relational database.”

MongoDB and Prisma

Bramer Schmidt said while a truism in software engineering is that code moves fast and data moves slow, but now we are starting to see more innovation around the data tooling space.

“And Mongo is a great example of that,” he said. “Mongo is a database that is much nicer to use for developers, you can express more different data constructs, and Mongo can handle things under the hood.”

Moreover, Prisma also is innovating around the developer experience for working with data, making it easier for developers to build applications that rely on data and do that faster, Bramer Schmidt said.

“The way we do that in Prisma is we have the tooling introspect your database, it will go and assemble documents in MongoDB, and then generate a schema based on that, and then it will pull that information into your development environment, such that you can, when you write queries, you will get autocompletion, and the IDE will tell you if you're making a mistake,” he said. “You will have that confidence in your environment instead of having to look at the documentation, try to remember what fields are where or how to do things. So that is increasing the confidence of the developer enabling them to move faster.

In an interview with The New Stack, renowned technologist Adrian Cockcroft discussed the process of fine-tuning Large Language Models (LLMs) through prompt engineering. Cockcroft, known for his roles at Netflix and Amazon Web Services, explained how to obtain tailored programming advice from an LLM. By crafting specific prompts like asking the model to provide code in the style of a certain expert programmer, such as Java's James Gosling, users can guide the AI's output.

Prompt engineering involves setting up conversations to bias the AI's responses. These prompts are becoming more advanced with plugins and loaded information that shape the model's behavior before use. Cockcroft highlighted the concept of fine-tuning, where models are adapted beyond what a prompt can contain. Companies are incorporating vast amounts of their internal data, like wiki pages and corporate documents, to train the model to understand their specific domain and processes.

Cockcroft pointed out the efficacy of ChatGPT within certain tasks, illustrated by his experience using it for data analysis and programming assistance. He also discussed the growing need for improved results from LLMs, which has led to the demand for vector databases. These databases store word meanings as vectors with associated weights, enabling fuzzy matching for enhanced information retrieval from LLMs. In essence, Cockcroft emphasized the multifaceted process of shaping and optimizing LLMs through prompt engineering and fine-tuning, reflecting the evolving landscape of AI-human interactions.

Learn more from The New Stack about LLMs and Prompt Engineering:

Top 5 Large Language Models and How to Use Them Effectively

The Pros (And Con) of Customizing Large Language Models

Prompt Engineering: Get LLMs to Generate the Content You Want

Developer Tips in AI Prompt Engineering

By now, almost everyone agreed platform engineering is probably a good idea, in which an organizations builds an internal development platform to empower coders and speed application releases. So, for this latest edition of The New Stack podcast,  we spoke with one of the pioneers in this space,  Zohar Einy, CEO of Port, to see how platform engineering would work in your organization. TNS Editor Joab Jackson hosted this conversation.

Port offers what it claims is the world's first low code platform for developers.

Rethinking Web Application Firewalls

With Port, an organization can build a software catalogue of approved tools, import its own data model, and set up workflows. Developers can consume all the resources they need through a self-service catalogue, without needing the knowledge how to set up a complex application, like Kubernetes. The DevOps and platform teams themselves maintain the platform.

Application owners aren't  the only potential users of a self-service catalogues, Einy points out in our convo. DevOps and system administration teams can also use the platform. A DevOps teams can set up automations "to make sure that [developers are] using the platform with the right mindset that fits with their organizational standards in terms of compliance, security, and performance aspects."

Even machines themselves could benefit from a self-service platform, for those who are looking to automate deployments as much as possible.

Einy offered an example: A CI/CD process could create a build process on its own. If it needs to check the maturity level of some tool, it can do so through an API call. If it's not adequately certified, the developer is notified, but if all the tools are sufficiently mature than the automated process can finish the build without further developer intervention.

Another possible process that could be automated would be the termination of permissions when their deadline has passed. Think about an early-warning system for expired digital certificates. "So it's a big driver for both for cost reduction and security best practices," Einy said.

Too Many Choices, Not Enough Code

But what about developer choice? Won't developers feel frustrated when barred from using the tools they are most fond of?

But this freedom to use any tool available was what led us to the current state of overcomplexity in full-stack development, Einy responded. This is why the role of "full-stack developer" seems like an impossible, given all the possible permutations at each layer of the stack.

Like the artist who finds inspiration in a limited palette, the developer should be able to find everything they need in a well-curated platform.

"In the past, when we talked about 'you-build-it-you-own-it', we thought that the developer needs to know everything about anything, and they have the full ownership to choose anything that they want. And they got sick of it, right, because they needed to know too much," Einy said. "So I think we are getting into a transition where developers are OK with getting what they need with a click of a button because they have so much work on their own."

In this conversation, we also discussed measuring success, the role of access control in DevOps, and open source Backstage platform, and its recent inclusion of paid plug-ins. Give it a listen!

Angel Diaz, Vice President of Technology, Capabilities, and Innovation at Discover Financial Services, spoke with TNS Host Alex Williams at the Open Source Summit in Vancouver, BC. Diaz emphasizes the importance of learning and collaboration among software engineers. He leads The Discover Technology Academy, a community of 15,000 engineers, which he describes as a place where craftsmen come together rather than an ivory tower institution.

Developers and engineers at Discover define and develop processes for software development. They start their journey by contributing atomic elements of knowledge, such as articles, blogs, videos, and tutorials, and then democratize that knowledge. Open source principles, communities, guilds, and established practices play a vital role in their work and discovery process.

Discover's developer experience revolves around the concept of the golden path, which goes beyond consuming content and includes aspects like code, automation, and setting up development environments. Pair programming and a cultural approach to learning are also incorporated into Discover's talent system.

Diaz highlights that Discover's work extends beyond their financial services company, as they share their knowledge and open source work with the external community through platforms like technology.discovered.com. This enables engineers to gain merit badges, such as maintainers or contributors, and showcase their expertise on professional platforms like LinkedIn.

Learn more at thenewstack.io

The Future of Developer Careers

Platform Engineer vs Software Engineer

How Donating Open Source Code Can Advance Your Career

In this episode of The New Stack Makers, technologist and author John Willis emphasized caution when considering AI solutions from vendors. He advised against blindly following vendor recommendations for "one-size-fits-all" AI products, likening it to discouraging learning Java in the past in favor of purchasing a product.

Willis stressed that DevOps serves as an example of how human expertise, not just products, solves problems. He urged C-level executives to first understand AI's intricacies and then make informed purchasing decisions, suggesting a "DevOps redo" to encourage experimentation and collaboration, similar to the early days of the DevOps movement.

Willis highlighted that early adopters of DevOps, like successful banks, heavily invested in developing their human capital. He cautioned against hasty product purchases, as the AI landscape is rife with startups that may quickly disappear or be acquired by larger companies.

Instead, Willis advocated for educating teams on effective data management techniques, including retrieval augmentation, to fine-tune large language models. He emphasized the need for data cleansing to build robust data pipelines and prevent LLMs from generating undesirable code or sensitive information.

According to Willis, the process becomes enjoyable when done correctly, especially for companies using LLMs at scale with retrieval augmentation. To ensure success, he suggested adding governance and structure, including content moderation and red-teaming of data, which vendors may not prioritize in their offerings.

Learn more from The New Stack about DevOps and AI:

AIOps: Is DevOps Ready for an Infusion of Artificial Intelligence?

How to Build a DevOps Engineer in Just 6 Months

Power up Your DevOps Workflow with AI and ChatGPT

The name "Falcon" for the UAE’s large language model (LLM) symbolizes the national bird's qualities of courage and perseverance, reflecting the vision of the Technology Innovation Institute (TII) in Abu Dhabi. TII, launched in 2020, addresses AI’s rapid advancements and unintended consequences by fostering an open-source approach to enhance community understanding and control of AI. In this New Stack Makers, Dr. Hakim Hacid, Executive Director and Acting Chief Researcher, Technology Innovation Institute emphasized the importance of perseverance and innovation in overcoming challenges. Falcon gained attention for being the first truly open model with capabilities matching many closed-source models, opening new possibilities for practitioners and industry. 

Last June, Falcon introduced a 40-billion parameter model, outperforming the LLaMA-65B, with smaller models enabling local inference without the cloud. The latest 180-billion parameter model, trained on 3.5 trillion tokens, illustrates Falcon’s commitment to quality and efficiency over sheer size. Falcon’s distinctiveness lies in its data quality, utilizing over 80% RefinedWeb data, based on CommonCrawl, which ensures cleaner and deduplicated data, resulting in high-quality outcomes. This data-centric approach, combined with powerful computational resources, sets Falcon apart in the AI landscape.

Learn more from The New Stack about Open Source AI: 

Open Source Initiative Hits the Road to Define Open Source AI 

 Linus Torvalds on Security, AI, Open Source and Trust

Transparency and Community: An Open Source Vision for AI 

Join our community of newsletter subscribers to stay on top of the news and at the top of your game. 

The pandemic has significantly accelerated the adoption of Kubernetes and cloud native environments as a way to accommodate the surge in remote workers and other infrastructure constraints. Following the beginning of the pandemic, however, organizations are retaining their investments for those organizations with cloud native infrastructure already in place. They have realized that cloud native is well worth maintaining their investments. Meanwhile, Kubernetes adoption continues to remain on an upward curve. And yet, challenges remain, needless to say. In this context, we look at the status of cloud native adoption, and in particular, Kubernetes at this time, compared to a year ago. 

In this episode of The New Stack Makers podcast, recorded on the floor of KubeCon + CloudNativeCon Europe 2022, we discussed these themes along with the state of Kubernetes and the community with James Laverack, staff solutions engineer, Jetstack a member of the Kubernetes release team, and Christoph Blecker, site reliability engineer, Red Hat, a member of the Kubernetes steering committee.

The amount of data created has doubled every year, presenting a host of challenges for organizations: security and privacy issues for starters, but also storage costs. What situations call for that data move to decentralized cloud storage rather than on-prem or even a single public cloud storage setup? What are the advantages and challenges of a decentralized cloud storage solution for data, and how can those be navigated?

On this episode of Makers, the New Stack podcast, Ben Golub, CEO of Storj, and Krista Spriggs, software engineering manager at the company, were joined by Alex Williams, founder and publisher of The New Stack, along with Heather Joslyn, TNS’ features editor. Golub and Spriggs talked about how decentralized storage for data makes sense for organizations concerned about cloud costs, security, and  resiliency.

"Developers aren't cryptographers. We can only do so much security training, and frankly, they shouldn't have to make hard choices about this encryption mode or that encryption mode. It should just, like, work," said Kenneth White,  a security principal at MongoDB, explaining the need for MongoDB's new Queryable Encryption feature. 

In this latest edition of The New Stack Makers podcast, we discuss [sponsor_inline_mention slug="mongodb" ]MongoDB[/sponsor_inline_mention]'s new end-to-end client-side encryption, which allows an application to query an encrypted database and keep the queries in transit encrypted, an industry first, according to the company.

White discussed this technology in depth to TNS publisher Alex Williams, in a conversation recorded at MongoDB World, held last week in New York.  

MongoDB has offered the ability to encrypt and decrypt documents since MongoDB 4.2, though this release is the first to allow an application to query the encrypted data. Developers with no expertise in encryption can write apps that use this capability on the client side, and the capability itself (available in preview mode for MongoDB 6.0) adds no noticeable overhead to application performance, so claims the company.

Data remains encrypted all times, even in memory and in the CPU; The keys never leave the application and cannot be accessed by the server. Nor can the database or cloud service administrator be able to look at the raw data.

For organizations, queryable encryption greatly expands the utility of using MongoDB for all sorts of sensitive and secret data. Customer service reps, for instance, could use the data to help customers with issues around sensitive data, such as social security numbers or credit card numbers.

In this podcast, White also spoke about the considerable engineering effort to make this technology possible — and make it easy to use for developers.

"In terms of how we got here, the biggest breakthroughs weren't cryptography, they were the engineering pieces, the things that make it so that you can scale to do key management, to do indexes that really have these kinds of capabilities in a practical way," Green said. 

It was necessary to serve a user base that needs maximum scalability in their technologies. Many have "monster workloads," he notes.

"We've got some customers that have over 800 shards, meaning 800 different physical servers around the world for one system. I mean, that's massive," he said. "So it was a lot of the engineering over the last year and a half [has been] to sort of translate those math and algorithm techniques into something that's practical in the database."

In her keynote address at the Linux Foundation's Open Source Summit Europe, Fatima Sarah Khalid emphasized that being an ally is more than just superficial gestures like wearing pronouns on badges or correctly pronouncing coworkers' names. True allyship involves taking meaningful actions to support and uplift individuals from underrepresented or marginalized backgrounds. This support is essential, not only in obvious ways but also in everyday interactions, which collectively create a more inclusive community.

Open source communities typically lack diversity, with only a small percentage of women, non-binary contributors, and individuals from underrepresented backgrounds. Khalid stressed the importance of improving diversity and inclusion through various means, including using inclusive language, facilitating asynchronous communication to accommodate global contributors, and welcoming non-technical contributions such as documentation.

Khalid also provided insights on making open source events more inclusive, like welcoming newcomers and marginalized groups, providing quiet spaces and enforcing a code of conduct, and partnering newcomers with mentors. Moreover, she highlighted GitLab's unique approach to allyship within the organization, including the Ally Lab, which pairs employees from different backgrounds to learn about and understand each other's experiences.

To encourage the audience to embrace allyship, Khalid shared a set of commitments to keep in mind, such as educating oneself about the experiences of marginalized groups, speaking up against inappropriate behavior, using one's voice to amplify marginalized voices, donating to support such groups, and advocating for equity and justice through social networks and connections. She also shared real-life examples of allyship, illustrating how meaningful actions can create positive change in communities.

Khalid's discussion with host Jennifer Riggins emphasizes the significance of meaningful, everyday actions to promote allyship in open source communities and organizations, ultimately contributing to a more diverse, inclusive, and equitable tech industry.

Learn more from The New Stack about Open Source, Allyship, and GitLab:

Embracing Open Source for Greater Business Impact

Leadership and Inclusion in the Open Source Community

How Implicit Bias Impacts Open Source Diversity and Inclusion

Investing in the Next Generation of Tech Talent

The whole world uses open source, but as we’ve learned from the Log4j debacle, “free” software isn’t really free. Organizations and their customers pay for it when projects aren’t frequently updated and maintained.

How can we support open source project maintainers — and how can we decide which projects are worth the time and effort to maintain?

“A lot of people pick up open source projects, and use them in their products and in their companies without really thinking about whether or not that project is likely to be successful over the long term,” Dawn Foster, director of open source community strategy at VMware’s open source program office (OSPO), told The New Stack’s audience during this On the Road edition of The New Stack’s Makers podcast.

In this conversation recorded at Open Source Summit Europe in Dublin, Ireland, Foster elaborated on the human cost of keeping open source software maintained, improved and secure —  and how such projects can be sustained over the long term.

The conversation, sponsored by Amazon Web Services, was hosted by Heather Joslyn, features editor at The New Stack.

Assessing Project Health: the ‘Lottery Factor’

One of the first ways to evaluate the health of an open source project, Foster said, is the “lottery factor”: “It's basically if one of your key maintainers for a project won the lottery, retired on a beach tomorrow, could the project continue to be successful?”

“And if you have enough maintainers and you have the work spread out over enough people, then yes. But if you're a single maintainer project and that maintainer retires, there might not be anybody left to pick it up.”

Foster is on the governing board for an project called Community Health Analytics Open Source Software — CHAOSS, to its friends — that aims to provide some reliable metrics to judge the health of an open source initiative.

The metrics CHAOSS is developing, she said, “help you understand where your project is healthy and where it isn't, so that you can decide what changes you need to make within your project to make it better.”

CHAOSS uses tooling like Augur and GrimoireLab to help get notifications and analytics on project health. And it’s friendly to newcomers, Foster said.

“We spend...a lot of time just defining metrics, which means working in a Google Doc and thinking about all of the different ways you might possibly measure something — something like, are you getting a diverse set of contributors into your project from different organizations, for example.”

Paying Maintainers, Onboarding Newbies

It’s important to pay open source maintainers in order to help sustain projects, she said. “The people that are being paid to do it are going to have a lot more time to devote to these open source projects. So they're going to tend to be a little bit more reliable just because they're they're going to have a certain amount of time that's devoted to contributing to these projects.”

Not only does paying people help keep vital projects going, but it also helps increase the diversity of contributors, “because you by paying people salaries to do this work in open source, you get people who wouldn't naturally have time to do that.

“So in a lot of cases, this is women who have extra childcare responsibilities. This is people from underrepresented backgrounds who have other commitments outside of work,” Foster said. “But by allowing them to do that within their work time, you not only get healthier, longer sustaining open source projects, you get more diverse contributions.”

The community can also help bring in new contributors by providing solid documentation and easy onboarding for newcomers, she said. “If people don't know how to build your software, or how to get a development environment up and running, they're not going to be able to contribute to the project.”

And showing people how to contribute properly can help alleviate the issue of burnout for project maintainers, Foster said:  “Any random person can file issues and bug maintainers all day, in ways that are not productive. And, you know, we end up with maintainer burnout...because we just don't have enough maintainers," said Foster.

“Getting new people into these projects and participating in ways that are eventually reducing the load on these horribly overworked maintainers is a good thing.”

Listen or watch this episode to learn more about maintaining open source sustainability.

AUSTIN, TEX. — How safe is the open source software that virtually every organization uses? You might not want to know, according to the results of a survey released by The Linux Foundation and Snyk, a cloud native cybersecurity company, at the foundation’s annual Open Source Summit North America, held here in June.

Forty-one percent of the more than 500 organizations surveyed don’t have high confidence in the security of the open source software they use, according to the research. Only half of participating companies said they have a security policy that addresses open source.

Furthermore, it takes more than double the number of days — 98 — to fix a vulnerability compared to what was reported in the 2018 version of the survey.

The research was conducted at the request of the Open Source Security Foundation (OpenSSF), a project of The Linux Foundation. For this On the Road episode of The New Stack Makers, Steve Hendrick, vice president of research at The Linux Foundation, and Matt Jarvis, director of developer relations at Snyk, were interviewed by Heather Joslyn, features editor at TNS.

Despite the alarming statistics, Jarvis cautions against treating all vulnerabilities as four-alarm fires, our guests said.

“Having a kind of zero-vulnerability target is probably unrealistic, because not all vulnerabilities are treated equal,” Jarvis said. Some “vulnerabilities” may not necessarily be a risk in your particular environment. It’s best to focus on the most critical threats to your network, applications and data.

One bright spot in the new report: Nearly one in four respondents said they’re looking for resources to help them keep their open source software — and all that depends on it — safe. Perhaps even more relevant to vendors: 62% of survey participants said they are looking to use more intelligent security-focused tools.

“There's a lot from a process standpoint that they are responsible for,” said Hendrick. “But they were very quick to jump on the bandwagon and say, we want the vendor community to do a better job at providing us tools, that makes our life a lot easier. Because I think everybody recognizes that solving the security problem is going to require a lot more effort than we're putting into it today.”

Jumping on the ‘SBOM Bandwagon’

Many organizations still seem confused about which of the dependencies the open source software they use has are direct and which are transitive (dependent on the dependencies), said Hendrick. One of the best ways to clarify things, he said, “ is to get on the SBOM bandwagon.”

Understanding an open source tool’s software bill of materials, or SBOM, is “going to give you great understanding of the components, it's going to give you usability, it's going to give you trust, you're gonna be able to know that the components are nonfalsified,” Hendrick said.

“And so that's all absolutely key from the standpoint of being able to deal with the whole componentization issue that is going on everywhere today.

Additional results from the research, in which core project maintainers discussed their best practices, will be released in the third quarter of 2022. Listen to the podcast to learn more about the report’s results and what Linux Foundation is doing to help upskill the IT workforce in cybersecurity.

No longer considered an ephemeral concept as it originally was, data management has become a huge issue and challenge, especially for managing stateless data in Kubernetes environments. 

Cloud Native Data Management Day at the recently held KubeCon + CloudNativeCon Europe 2021 event in May and the state of data management were the subject of discussion in this edition of The New Stack Makers podcast, hosted by Alex Williams, founder and publisher of The New Stack. The guests were Michael Cade, senior global technologist, Veeam Software and Nigel Poulton, owner of nigelpoulton.com, which offers Kubernetes and Docker training and other services. Both Cade and Poulton were also involved in the organization of Cloud Native Data Management Day.

VALENCIA, Spain —  WebAssembly (Wasm) is among the more hot topics under the CNCF project umbrella.  In this episode of The New Stack Makers podcast, recorded on the show floor of KubeCon + CloudNativeCon Europe 2022, Liam Randall, CEO and co-founder, Cosmonic, and Colin Murphy, senior software engineer, Adobe, discuss why Wasm’s future looks bright. 

A quintessential feature of Wasm is that it functions on a CPU level, not unlike Java or Flash. This means, Randall said, that Wasm “can run anywhere.” “Everybody can start using Wasm, which functionally works like a tiny CPU. You can even put WebAssembly inside other applications.”

The fact that Wasm has a binary format (with .wasm file format) and can be used to run on a CPU level like C or C++ does means it is highly portable. “WebAssembly really is exciting because it gives us two fundamental things that are truly amazing: One is portability across a diverse set of CPUs and architectures, and even portability into other places, like into a web browser,” said Randall. “It also gives us a security model that's portable, and works the same across all of those different landscape settings.”

This portability makes wasm an excellent candidate for edge applications. Its inference capabilities for machine learning (ML) at the edge are particularly promising for applications distributed across many different applications, Murphy described. Wasm is also particularly apt for collaboration for ML edge and other applications. “Collaborative experiences are what WebAssembly is really perfectly in position for," he continued.

In many ways, the name “WebAssembly” is not intuitively reflective of its meaning. “WebAssembly is neither web nor assembly — so, it's a somewhat awkwardly named technology, but a technology that is worth looking into,” Randall said. “There are incredible opportunities for your internal teams to transform the way they do business to save costs and be more secure by adopting this new standard.”

Amazon Web Services (AWS) has introduced PG Vector, an open-source tool that integrates generative AI and vector capabilities into PostgreSQL databases. Sirish Chandrasekaran, General Manager of Amazon Relational Database Services, explained at Open Source Summit 2024 in Seattle that PG Vector allows users to store vector types in Postgres and perform similarity searches, a key feature for generative AI applications. 

The tool, developed by Andrew Kane and offered by AWS in services like Aurora and RDS, originally used an indexing scheme called IVFFlat but has since adopted Hierarchical Navigable Small World (HNSW) for improved query performance. 

HNSW offers a graph-based approach, enhancing the ability to find nearest neighbors efficiently, which is crucial for generative AI tasks. AWS emphasizes customer feedback and continuous innovation in the rapidly evolving field of generative AI, aiming to stay responsive and adaptive to customer needs.

Learn more from The New Stack about Vector Databases 

Top 5 Vector Database Solutions for Your AI Project 

Vector Databases Are Having a Moment – A Chat with Pinecone 

Why Vector Size Matters 

Join our community of newsletter subscribers to stay on top of the news and at the top of your game. https://thenewstack.io/newsletter/

What is Terraform?

Terraform is HashiCorp’s flagship software. The open source tool provides a way to define IT resources — such as monitoring software or cloud services — in human-readable configuration files. These files, which serve as blueprints, can then be used to automatically provision the systems themselves. Kubernetes deployments, for instance, can be streamlined through Terraform.

"Terraform basically translates what your configuration was codified in by your configuration, and provisions it to that desired end state," explained Meghan Liese, [sponsor_inline_mention slug="hashicorp" ]HashiCorp[/sponsor_inline_mention] vice president of product and partner marketing in this podcast and video recording, recorded at the company's user conference, HashiConf 2022, held this month in Los Angeles.

For this interview, Liese discusses the latest enhancements to Terraform, and Terraform Cloud, a managed service offering that is part of the HashiCorp Cloud Platform.

[Embed Podcast]

Why Should Developers be Interested in Terraform?

Typically, the DevOps teams, or system administrators, use Terraform to provision infrastructure, but there is also growing interest to allow developers to do it themselves, in a self-service fashion, Liese explained. Multicloud skills are in short supply, concluded the 2022 HashiCorp State of Cloud Strategy Survey, so making the provision process easier could help more developers, the company reckons.

A Terraform self-service model, which was introduced earlier this year, could “cut down on the training an organization would need to do to get developers up to speed on using the infrastructure-as-code software,” Liese said.

In this “no code” setup, developers can pick from a catalog of no-code-ready modules, which can be deployed directly to workspaces. No need to learn the HCL configuration language. And the administrators will no longer have to answer the same “how-do-I-do-this-in-HCL?” queries.

The new console interface aims to greatly expand the use of Terraform.  The company has been offering self-service options for a while, by way of an architecture that allows for modules to be reused through the private registry for Terraform Cloud and Terraform Enterprise.

What is the Make Code Block and Why is it Important?

The recent release of Terraform 1.3 came with the promise to greatly reduce the amount of code HCL jockeys must manage, through the improvement of the make code block.

Actually, make has been available since Terraform 1.1, but some kinks were worked out for this latest release. What make does is provide the ability to refactor resources within a Terraform configuration file, moving large code blocks off as separate modules, where they can be discovered through a public or private registry.

What is Continuous Validation?

With the known state of a system captured on Terraform, it is a short step to check to ensure that the actual running system is identical to the desired state captured in HCL. Many times “drift” can occur, as administrators, or even the apps themselves, make changes to the system. Especially in regulated environments, such as hospitals, it is essential that a system is in a correct state.

Earlier this year, HashiCorp added Drift Detection to Terraform Cloud to continuously check infrastructure state to detect changes and provide alerts and offer remediation if that option is chosen. Now, another update, Continuous validation expands these checks to include user assertions, or post-conditions, as well.

One post-condition may be something like ensuring that certificates haven’t expired. If they do, the software can offer an alert to the admin to update the certs. Another condition might be to check for new container images, which may have been updated as a response to a security patch.

The story goes something like this:

There's this marketing manager who is trying to time a launch. She asks the developer team when the service will be ready. The dev team says maybe a few months. Let's say three months from now in April. The marketing manager begins prepping for the release.

The dev team releases the services the following week.

It's not an uncommon occurrence.

Edith Harbaugh is the co-founder and CEO of LaunchDarkly, a company she launched in 2014 with John Kodumal to solve these problems with software releases that affect organizations worldwide. Today, LaunchDarkly has 4,000 customers and an annual return revenue rate of $100 million.

We interviewed Harbaugh for our Tech Founder Odyssey series on The New Stack Makers about her journey and LaunchDarkly's work. The interview starts with this question about the timing of dev releases and the relationship between developers and other constituencies, particularly the marketing organization.

LaunchDarkly is the number one feature management company, Harbaugh said. "Their mission is to provide services to launch software in a measured, controlled fashion. Harbaugh and Kodumal, CTO, founded the company on the premise that software development and releasing software is arduous.

"You wonder whether you're building the right thing," Harbaugh said, who has worked as both an engineer and a product manager. "Once you get it out to the market, it often is not quite right. And then you just run this huge risk of how do you fix things on the fly."

Feature flagging was a technique that a lot of software companies did. Harbaugh worked at Tripit, a travel service, where they used feature flags as did companies such as Atlassian, where Kodumal had developed software.

"So the kernel of LaunchDarkly, when we started in 2014, was to make this technique of feature flagging into a movement called feature management, to allow everybody to build better software faster, in a safer way."

LaunchDarkly allows companies to release features however granular an organization wants, allowing a developer to push a release into production in different pieces at different times, Harbaugh said. So, a marketing organization can send a release out even after the developer team has released it into production.

"So, for example, if, we were running a release, and we wanted somebody from The New Stack to see it first, the marketing person could turn it on just for you."

Harbaugh describes herself as a huge geek. But she also gets it in a rare way for geeks and non-geeks alike. She and Kodumal took a concept used effectively by develops, transforming it into a service that provides feature management for a broader customer base, like the marketer wanting to push releases out in a granular way for a launch on the East Coast that is pre-programmed with feature flags in advance from the company office the previous day in San Francisco.

The idea is novel, but like many intelligent, technical founders, Harbaugh's journey reflects her place today. She's a leader in the space, and a fun person to talk to, so we hope you enjoy this latest episode in our tech founder series from The New Stack Makers.

There is much discussion about technology and tool gaps when organizations make the shift to cloud environments. However, a major — and often less-discussed — challenge is how to ensure that the DevOps team has the necessary skillsets to see the project through. Making sure that the right in-house talent and DevSecOps culture is in place to make the shift without exposing the organization's data and applications to security risks is especially critical.

In this The New Stack Makers podcast hosted by Alex Williams, founder and publisher of TNS,  guest Ashley Ward, technical director, office of the CTO, Palo Alto Networks, discussed the associated DevSecOps skillsets challenges for cloud deployments.

Since the release of OpenAI's ChatGPT-3 in late 2022, various industries have been actively exploring its applications. Madhukar Kumar, CMO of SingleStore, discussed his experiments with large language models (LLMs) in this podcast episode with TNS host Heather Joslyn. He mentioned a specific LLM called Gorilla, which is trained on APIs and can generate APIs based on specific tasks. Kumar also talked about SingleStore Now, an AI conference, where they plan to teach attendees how to build generative AI applications from scratch, focusing on enterprise applications.

Kumar highlighted a limitation with current LLMs - they are "frozen in time" and cannot provide real-time information. To address this, a method called "retrieval augmented generation" (RAG) has emerged. SingleStore is using RAG to keep LLMs updated. In this approach, a user query is first matched with up-to-date enterprise data to provide context, and then the LLM is tasked with generating answers based on this context. This method aims to prevent the generation of factually incorrect responses and relies on storing data as vectors for efficient real-time processing, which SingleStore enables.

This strategy ensures that LLMs can provide current and contextually accurate information, making AI applications more reliable and responsive for enterprises.

Learn more from The New Stack about LLMs and SingleStore:

Top 5 Large Language Models and How to Use Them Effectively

Using ChatGPT for Questions Specific to Your Company Data

6 Reasons Private LLMs Are Key for Enterprises

Wix is a cloud-based development site for making HTML 5 websites and mobile sites with drag and drop tools. It is suited for the beginning user or the advanced developer, said Hila Fish, senior DevOps engineer for Wix, in an interview for The New Stack Makers at HashiCorp’s HashiConf Global conference in Los Angeles earlier this month.

Our questions for Fish focused on Terraform, the open source infrastructure-as-code software tool:

• How has Terraform evolved in uses since Fish started using it in 2018?
• How does Wix make the most of Terraform to scale its infrastructure?
• What are some best practices Wix has used with Terraform?
• What are some pitfalls to avoid with Terraform?
• What is the approach to scaling across teams and avoiding refactoring to keep the integrations elegant and working

Fish started using Terraform in an ad-hoc manner back in 2018. Over time she has learned how to use it for scaling operations.

“If you want to scale your infrastructure, you need to use Terraform in a way that will allow you to do that,” Fish said. 

Terraform can be used ad-hoc to create a machine as a resource, but scale comes with enabling infrastructure that allows the engineers to develop templates that get reused across many servers.

“You need to use it in a way that will allow you to scale up as much as you can,” Fish said.

Fish said best practices come from how to structure the Terraform code base.

Much of it comes down to the teams and how Terraform gets implemented. Engineers each have their way of working. Standard practices can help. In onboarding new teams, a structured code base can be beneficial. New teams onboard and use models already in the code base.

And what are some of the pitfalls of using Terraform?

We get to that in the recording and more about integrations, why Wix is still on version 0.13, and some new capabilities for developers to use Terraform.

Users have historically needed to learn HashiCorp configuration language (HCL) to use the HashiCorp configuration language. At Wix, Fish said, the company is implementing Terraform on the backend with a UI that developers can use without needing to learn HCL.

If there is a secret to the success of TypeScript, it is in the type checking, ensuring that the data flowing through the program is of the correct kind of data. Type checking cuts down on errors, sets the stage for better tooling, and allows developers to map their programs at a higher level. And TypeScript itself, a statically-typed superset of JavaScript, ensures that an army of JavaScript programmers can easily enjoy these advanced programming benefits with a minimal learning curve.

In this latest edition of The New Stack Makers podcast, we spoke with a few of TypeScript's designers and maintainers to learn a bit more about the design of the language: Ryan Cavanaugh, a principal software engineering manager for Microsoft; Luke Hoban, chief technology officer for Pulumi, who was one of original creators of TypeScript, and; Daniel Rosenwasser, Senior Program Manager, Microsoft. TNS editors Darryl Taft and Joab Jackson hosted the discussion.

At Cloud Native Security Con, we sat down with Solo.io's Marino Wijay and Jim Barton, who discussed how service mesh technologies have matured, especially now with the removal of sidecars in Ambient Mesh that it developed with Google.

Ambient Mesh is "a new proxy architecture that, according to the Solo.io site, "moves the proxy to the node level for mTLS and identity. It also allows a policy-enforcement policy to manage Layer 7 security filters and policies.

A sidecar is a mini-proxy, a mini-firewall, like an all-in-one router, said Wijay, who does developer relations and advocacy for Solo. A sidecar receives instructions from an upstream control plane.

"Now, one of the things that we started to realize with different workloads and different patterns of communication is that not all these workloads need a sidecar or can take advantage of the sidecar," Wijay said. "Some better operate without the sidecar."

Ambient Mesh reflects the maturity of service mesh and the difference between day one and day two operations, said Barton, a field engineer with Solo.

"Day one operations are a lot about understanding concepts, enabling developers, initial configurations, that sort of thing," Barton said. "The community is really much more focused and Ambient Mesh is a good example of this on day two concerns. How do I scale this? How do I make it perform in large environments? How can I expand this across clusters, clusters in multiple zones in multiple regions, that sort of thing? Those are the kinds of initiatives that we're really seeing come to the forefront at this point."

With the maturity of service mesh comes the users. In the context of security, that means the developer security operations person, Barton said. It's not the developer's job to connect services. Their job is to build out the services.

"It's up to the platform operator, or DevSecOps engineers to create that, that fundamental plane or foundation for where you can deploy your services, and then provide the security on top of it," Barton said.

The engineers then have to configure it and think it through. "How do I know who's doing what and who's talking to who, so that I can start forming my zero trust posture?," Barton said.

In this episode of The New Stack Makers podcast, Uma Daniel, a product manager at UST, discusses the current complexities in the global economy, marked by low unemployment except in the tech industry, high inflation, high interest rates, a volatile stock market, and the looming threat of recession. Amid these challenges, organizations are seeking ways to enhance their operational efficiency.

Daniel introduces the concept of FinOps, which goes beyond just managing cloud costs. Instead, it focuses on leveraging the cloud to generate revenue. This represents a cultural shift in many organizations, emphasizing the need for a mindset change across different departments, including business, finance, and procurement.

She dispels misconceptions, such as the belief that only certain teams should be involved in the FinOps process. Daniel stresses that it's a collaborative effort involving various teams, and it's best to adopt FinOps at the beginning of a cloud journey. Once an organization is already established in the cloud, implementing FinOps becomes more challenging.

To foster collaboration, Daniel suggests identifying team members willing to champion FinOps and forming cross-functional teams to lead the initiative. Regular committee meetings and the establishment of generic policies, such as project budgets, help control cloud spending.

This episode, hosted by Heather Joslyn, provides insights into how to initiate and implement a FinOps strategy and highlights common ways in which organizations waste cloud resources.

Learn more from The New Stack about FinOps and UST:

Cloud Cost-Unit Economics — A Modern Profitability Model

What Is FinOps? Understanding FinOps Best Practices for Cloud

Very Large Enterprises Need a Different Approach to FinOps

No matter how much we prepare, deployments don’t always go as planned. In this edition of The New Stack Makers podcast, hosted by Alex Williams, founder and publisher of The New Stack, Isabelle Miller, software engineer, LaunchDarkly, describes how DevOps teams can build processes to help remove unwanted surprises during release cycles —  and why they do not need to be stressful.

One of the main things Miller said she has discovered since joining LaunchDarkly at the beginning of 2020 is the importance of having procedures in place for when things do go wrong, “because things are going to go wrong,” she said.

“You need to be able to manage that problem as quickly as possible, and minimize any harm before things get out of control when that happens,” said Miller. “So, one of the great things about working at LaunchDarkly is that I get to use our products. And one of the wonderful things about LaunchDarkly’s feature flags is that you can just turn things off.”

Explore the complex intersection of AI and open source with insights from experts in this illuminating discussion. Amanda Brock, CEO of OpenUK, reveals the challenges in labeling AI as open source amidst legal ambiguities. The dialogue, led by TNS host Alex Williams, delves into the evolution of open source licensing, its departure from traditional models, and the complications arising from applying open source principles to AI, which encompasses sensitive data governed by privacy laws.

The focus turns to "Llama 2," a contentious example where Meta labeled their language model as open source, sparking confusion. Notable guests Erica Brescia, Managing Director at Redpoint Ventures, and Steven Vaughan-Nichols, founder of Open Source Watch, weigh in on this topic. Brock emphasizes that AI's complexity prevents it from aligning with the Open Source Definition, necessitating a clear distinction between open innovation and open source.

Amidst these debates, the Open Source Initiative (OSI) is crafting a new definition tailored for AI, sparking anticipation and discussion about its implications. The necessity for an evolved understanding of open source and its licenses is underscored, as the rapid evolution of technology challenges established norms. The journey concludes with reflections on vendors transitioning from open source licenses to Server Side Public License (SSPL) due to cloud-related considerations, raising questions about the future of open source in a dynamically changing tech landscape.

Learn more from The New Stack about open source and AI:

Open Source May Yet Eat Google's and OpenAI's AI Lunch

Open Source Movement Emerging in AI To Counter Greed

How AI Can Learn from the Struggles of Open Source

"Many Ukrainians continue working. A very good opportunity is to continue working with them, to buy Ukrainian software products, to engage with people who are working [via] UpWork. Help Ukrainians by giving them the ability to work, to do some paid work," whether still in the country or as refugees abroad. If you take something from this conversation, Anastasiia Voitova's words may be the ones that should stick. After all, Ukraine has a renowned IT workforce, with IT outsourcing among its most important exports.

Voitova, the head of customer solutions and security software engineer at Cossack Labs, just grabbed her laptop and some essentials when she suddenly fled to the mountains last month to "a small village that doesn't even have a name." She doesn't have much with her, but she has more work to do than ever — to meet her clients' increasing demand for cybersecurity defenses and to support the Ukrainian defense effort. Earlier this month, her Ukraine-based team even released a new open source cryptographic framework for data protection, on time, amidst the war.

Voitova was joined in this episode of The New Stack Makers by Oleksii Holub, open source developer, software consultant and GitHub Star, and Denys Dovhan, front-end engineer at Wix. All three of them are globally known open source community contributors and maintainers. And all three had to suddenly relocate from Kyiv this February. This conversation is a reflection into the lives of these three open source community leaders during the first three weeks of the Russian invasion.

This conversation aims to help answer what the open source community and the tech community as a whole can do to support our Ukrainian colleagues and friends. Because open source is a community first and foremost. 

"Open source for me is a very big part of my life. Idon't try to like gain anything out of it, I just code things. If I had a problem, I solve it, and I think to myself, why not share it with other people," Holub said.

He sees open source as an opportunity for influence in this war, but also is acutely aware that his unpaid labor could be used to support the aggression against his country. That's why he added terms of use to his open source projects that use of his code implicitly means you condemn the Russian invasion. This may be controversial in the strict open source licensing world, but the semantics of OSS seem less important to Holub right now.

Of course, when talking about open source, the world's largest code repository GitHub comes up. Whether GitHub should block Russia is an on going OSS debate. On the one hand, many are concerned about further cutting off Russia — which has already restricted access to Facebook, Instagram, and Twitter — from external news and facts about the ongoing conflict. On the other hand, with the widespread adoption of OSS in Russia, it's reasonable to assume swaths of open source code is directly supporting the invasion or at least supporting the Russian government through income, taxes, and some of the Kremlin's technical stack.

For Dovhan, there's a middle ground. His employer, website builder Wix, has blocked all payments in Russia, but has maintained its freemium offering there. "There is no possibility to pay for your premium website. But you still can make a free one, and that's a possibility for Russians to express themselves, and this is a space for free speech, which is limited in Russia." He proposes that GitHub similarly allows the creation of public repos in Russia, but that it blocks payments and private repos there.

Dovhan continued that "I believe [the] open source community is deeply connected and blocking access for Russian developers, might cause serious issues in infrastructure. Alot of projects are actually made by Russian developers, for example, PostCSS, Nginx, and PostHTML."

These conversations will continue as this war changes the landscape of the tech world as we know it. One thing is for sure, Voitova, Dovhan and Holub have joined the hundreds of thousands of Ukrainian software developers in making a routine of work-war balance, doing everything they can, every waking hour of the day.

VALENCIA – Open source code is part of at least 70% of enterprise stacks. Yet, a lot of open source contributors are still unpaid volunteers. Even more than tech as a whole, the future of open source relies on the community. Unless you're among the top tier funded open source projects, your sustainability replies on building a community – whether you want to or not – and cultivating project leadership to help recruit new maintainers – whether you want to hand over the reins or not.

That's where the Tech Advisory Group or TAG on Contributor Strategy comes in, acting as maintainer relations for the Cloud Native Computing Foundation. In this episode of The New Stack Makers podcast, recorded on the floor of KubeCon + CloudNativeCon Europe 2022, we talk to Dawn Foster, VMware's director of open source community strategy; Josh Berkus, Red Hat's Kubernetes community manager; Catherine Paganini, Bouyant's head of marketing and community; and Deepthi Sigireddi, a software engineer at PlanetScale. Foster and Berkus are the co-chairs of the Contributor Strategy TAG, while Paganini is the creator of Linkerd and Sigireddi is a maintainer of Vitess, both CNCF graduated projects. Each brought their unique experience in both open source contribution and leadership to talk about the open source contributor experience, sustainability, governance, and guidance.

With 65% of KubeConEU attendees at a CNCF event for the first time, albeit still during a pandemic, it makes for an uncertain signal for the future of open source. It either shows that there's a burst of interest for newcomers or that there's a dwindling interest in long-term contributions. The executive director of CNCF Priyanka Sharma even noted in her keynote that contributions for the foundation's biggest project Kubernetes have grown stagnant.

"I see it as a positive thing. I think it's always good to get some new blood into the community. And I think you know, the projects are working to do whatever they can to get new contributors," Foster said.

[sponsor_note slug="kubecon-cloudnativecon" ][/sponsor_note]

But it's not just about how many contributors but who. One thing that was glaringly apparent at the event was the lack of diversity, with the vast majority of the 7,000 KubeConEU participants being young, white men. This isn't surprising at all, as open source is still based on a lot of voluntary work which naturally excludes those most marginalized within the tech industry and society, which is why, according to GitHub's State of the Octoverse, it sees only about 4% women and nonbinary contributors, and only about 2% from the African continent.

If open source is such an integral part of tech's future, that future is built with more inequity than ever before.

"The barrier to entry to open source right now is having free time. And to do free work? Yes, and let's face it, women still do a lot of childcare, a lot of housework, much more than men do, and they have less free time." Sigireddi continued that there are other factors which discourage those widely underrepresented in tech from participating, including "not having role models, not seeing people who look like you, the communities tend to have in-jokes [and other] things that are cultural, which minorities may not be able to relate to." Most open source code, while usually forked globally, exists in English only.

One message throughout KubeConEU was, if a company relies on an open source project, it should pay some of its staff to contribute to and support that project because business may depend on it. This will in turn help bring OSS up a bit closer to the standard of the still abysmal tech industry statistics.

"I think from an ecosystem perspective, I think that companies paying people to do the work on open source makes a big difference," Foster said. "At VMware, we pay lots of people who work primarily on upstream open source projects. And I think that does help us get more diversity into the community, because then people can do it as part of their regular day jobs."

Encouraging those contributors that are underrepresented in OSS to speak up and be more representative of projects is another way to attract more diverse contributors. Berkus said the Contributors Strategy TAG had a meeting at KubeConEU with a group of primarily Italian women who have started in inclusiveness effort, starting with some things like speaker coaching and placement.

"It turns out that a lot of things that you need to do to have more diverse contributors are things you actually needed to do anyway, just to make things better for all new contributors," Berkus explained.

Indeed, welcoming new open source contributors – at all levels and in both technical and non-technical roles – is an important focus of the TAG. Paganini, along with colleague Jason Morgan, is co-author of the CNCF Landscape Guide, which acts as a welcome to the massive, overwhelming cloud native landscape. What she has found is that people will use the open source technology, but they will contribute to it because of the community.

"We see a lot of projects really focusing on code and docs, which of course is the basics, but people don't come for the technology per se. You can have the best technology, it's amazing, and people are super excited, but if the community isn't there, if they don't feel welcome," they won't stick around, Paganini said. "People want to be part of a tribe, right?"

Then, once you've successfully recruited and onboarded your community, you've got to work to not only retain but promote from within. All this and more is jam-packed into this lively discussion that cannot be missed!

More on open source diversity and inclusion efforts:

 

• Beat Affinity Bias with Open Source Diversity and Inclusion

 

• Open Source Communities Need More Safe Spaces and Codes of Conducts. Now.

 

• WTF is Wrong with Open Source Communities

 

• Look Past the Bros, and Concerns About Open Source Inclusion Remain

 

• How to Give and Receive Technical Help in Open Source Communities

 

• Navigating the Messy World of Open Source Contributor Data

 

• How to Find a Mentor and Get Started in Open Source

TNS host Alex Williams is joined by Florian Valeye, a data engineer at Back Market, to shed light on the evolving landscape of data engineering, particularly focusing on Delta Lake and his contributions to open source communities. As a member of the Delta Lake community, Valeye discusses the intersection of data warehouses and data lakes, emphasizing the need for a unified platform that breaks down traditional barriers.

Delta Lake, initially created by Databricks and now under the Linux Foundation, aims to enhance reliability, performance, and quality in data lakes. Valeye explains how Delta Lake addresses the challenges posed by the separation of data warehouses and data lakes, emphasizing the importance of providing asset transactions, real-time processing, and scalable metadata.

Valeye's involvement in Delta Lake began as a response to the challenges faced at Back Market, a global marketplace for refurbished devices. The platform manages large datasets, and Delta Lake proved to be a pivotal solution in optimizing ETL processes and facilitating communication between data scientists and data engineers.

The conversation delves into Valeye's journey with Delta Lake, his introduction to Rust programming language, and his role as a maintainer in the Rust-based library for Delta Lake. Valeye emphasizes Rust's importance in providing a high-level API with reliability and efficiency, offering a balanced approach for developers.

Looking ahead, Valeye envisions Delta Lake evolving beyond traditional data engineering, becoming a platform that seamlessly connects data scientists and engineers. He anticipates improvements in data storage optimization and envisions Delta Lake serving as a standard format for machine learning and AI applications.

The conversation concludes with Valeye reflecting on his future contributions, expressing a passion for Rust programming and an eagerness to explore evolving projects in the open-source community. 

Learn more from The New Stack about Delta Lake and The Linux Foundation:

Delta Lake: A Layer to Ensure Data Quality

Data in 2023: Revenge of the SQL Nerds

What Do You Know about Your Linux System?

TechWorld with Nana is one of the most popular resources for people looking to get into or progress a DevOps career. Nana Janashia, the creator of TechWorld with Nana, is a DevOps trainer and consultant who joined us to discuss why DevOps is needed now more than ever and how this is the perfect time to begin a career in DevOps.

Host Alex Williams and Nana go over the key concepts of DevOps. Then they talk about how the complexity of tools can sidetrack and complicate the learning process for those new to DevOps and why focusing on concepts rather than tools the way to go. Before wrapping up the conversation, they even talk about the best ways for people to get involved who are new to DevOps.

Nana's journey into DevOps commenced during her time as an engineer in Austria, where she began exploring Kubernetes. As inquiries from colleagues poured in, she recognized her knack for demystifying complex topics, catalyzing her passion for teaching. Viewers attest to switching to DevOps careers after watching her videos.

Throughout the conversation, we learned how people can discover the world of DevOps through TechWorld with Nana as an expert guide. With a large YouTube audience, online courses, workshops, and corporate training, Nana has empowered countless individuals in advancing their DevOps expertise. The six-month boot camps from TechWorld with Nana encompass a comprehensive curriculum, starting with fundamentals and culminating in hands-on programming abilities, Python automation, configuration management, and Prometheus-based monitoring.

Nana underscores that DevOps, still a relatively nascent profession, suffers from role ambiguity both among engineers and within companies aspiring to implement it. This confusion stems from differing workflows and environments when engineers switch jobs. Nana's insights bring clarity to these challenges, acknowledging the evolving chaos of the DevOps culture and its driving force for innovation in managing intricate distributed technologies.

Learn more about DevOps from TNS, Roadmap (our sister site), and TechWorld with Nana:

TechWorld with Nana - DevOps  Bootcamp

TechWorld with Nana - DevSecOps Bootcamp

DevOps Learning Roadmap

DevOps News, Trends, and Analysis

An organization that has any ambitions or hopes to scale application deployments across cloud native environments is not going to get very far without automation.

From CI/CD support, increasing application deployment speed  — often across different environments — and maintaining compliance and security, operations teams manually managing these processes is just not humanly possible after a certain point.

In this latest episode of The New Stack Makers podcast,  Abby Kearns, Chief Technology Officer and head of R&D, and Chip Childers, Puppet Chief Architect, discussed what automation for infrastructure management for cloud native deployments means for Puppet and for the IT industry. Alex Williams, founder and publisher of TNS, hosted this interview.

Google’s open source program certainly has come a long way since 2003. That was when the search engine giant could still arguably be called a startup, Android had not yet been acquired and open source projects Kubernetes, Go and Chromium were years away in the making.

It was also then that Google co-founders Larry Page and Sergey Brin asked their favorite recruiter to go and find an “open source person,” recounted Chris DiBona, the company’s director for open source.  Already an open source pioneer before joining Google, DiBona continues to oversee the tech giant’s open source program, which continues to have major implications for the IT industry and the open source community.

In this New Stack Makers podcast, DiBona discusses Google’s open source policy, as well as the search engine giant’s plans for its open source future.  Alex Williams, founder and publisher of The New Stack, hosted this podcast.

Debate continues in the industry about what observability is, and more specifically, what it should offer DevOps, especially those working in operations who are often responsible for detecting those “unknown unknowns.” In this The New Stack Makers podcast hosted by Alex Williams, founder and publisher of The New Stack, Bartek Plotka, a principal engineer at Red Hat, a SIG observability tech Lead for Thanos and a Prometheus maintainer; and Richard Hartmann, community director at Grafana, a Prometheus maintainer, OpenMetrics founder and a CNCF SIG observability chair member, discuss how observability should be easier to use and how it can be cost effective.

Observability in multi-cloud environments is becoming increasingly complex, as highlighted by Martin Mao, CEO and co-founder of Chronosphere. This challenge has two main components: a rise in customer-facing incidents, which demand significant engineering time for debugging, and the ineffectiveness and high cost of existing tools. These issues are creating a problematic return on investment for the industry.

Mao discussed these observability challenges on The New Stack Makers podcast with host Heather Joslyn, emphasizing the need to help teams prioritize alerts and encouraging a shift left approach for security responsibility among developers. With the adoption of distributed cloud architectures, organizations are not only dealing with a surge in data but also facing a cultural shift towards DevOps, where developers are expected to be more accountable for their software in production.

Historically, operations teams handled software in production, but in the cloud-native world, developers must take on these responsibilities themselves. Many current observability tools were designed for centralized operations teams, which creates a gap in addressing developer needs.

Mao suggests that cloud-native observability tools should empower developers to run and maintain their software in production, providing insights into the complex environments they work in. Moreover, observability tools can assist developers in understanding the intricacies of their software, such as its dependencies and operational aspects.

To streamline the data obtained from observability efforts and manage costs, Chronosphere introduced the "Observability Data Optimization Cycle." This framework starts with establishing centralized governance to set budgets for teams generating data. The goal is to optimize data usage to extract value without incurring unnecessary costs. This approach applies financial operations (FinOps) concepts to the observability space, helping organizations tackle the challenges of cloud-native observability.

Learn more from The New Stack about Observability and Chronosphere:

Observability Overview, News and Trends

4 Key Observability Best Practices

Top Ways to Reduce Your Observability Costs

Top 4 Factors for Cloud Native Observability Tool Selection

This New Stack Makers podcast co-hosted by TNS founder and publisher, Alex Williams and Joan Westenberg, founder and writer of Joan’s Index, discussed Copilot. Westenberg highlighted its integration with Microsoft 365 and its role as a coding assistant, showcasing its potential to streamline various tasks. 

However, she also revealed its limitations, particularly in reliability. Despite being designed to assist with tasks across Microsoft 365, Copilot's performance fell short during Westenberg's tests, failing to retrieve necessary information from her email and Microsoft Teams meetings. While Copilot proves useful for coding, providing helpful code snippets, its effectiveness diminishes for more complex projects. Westenberg's demonstrations underscored both the strengths and weaknesses of Copilot, emphasizing the need for improvement, especially in reliability, to fulfill its promise as a versatile work companion.

Learn more from The New Stack about Copilot 

Microsoft One-ups Google with Copilot Stack for Developers 

Copilot Enterprises Introduces Search and Customized Best Practices 

Join our community of newsletter subscribers to stay on top of the news and at the top of your game. 

The mystery and miracle of flight sparked Ev Kontsevoy’s interest in engineering as a child growing up in the Soviet Union.

“When I was a kid, when I saw like airplane flying over, I was having a really hard time not stopping and staring at it until it's gone,” said Kontsevoy, co-founder and CEO of Teleport, said in this episode of the Tech Founders Odyssey podcast series. “I really wanted to figure out how to make it fly.”

Inevitably, he said, the engineering path led him to computers, where he was thrilled by the power he could wield through programming. “You're a teenager, no one really listens to you yet, but you tell a computer to go print number 10 ... and then you say, do it a million times. And the stupid computer just prints 10 million. You feel like a magician that just bends like machines to your will.”

In this episode of the series, part of The New Stack Makers podcast, Kontsevoy discussed his journey to co-founding Teleport, an infrastructure access platform, with TNS co-hosts Colleen Coll and Heather Joslyn.

The adoption of GitOps, improvements to APIs and the increasing reach of virtual machine language WebAssembly (Wasm) are influencing the developer experience, and ultimately, how DevOps teams reach their application-deployment and -management goals. These were among the more talked-about themes at Cloud Native Computing Foundation KubeCon + CloudNativeCon EU

Putting it all into context, Alex Williams, founder and publisher, and Joab Jackson, managing editor, of The New Stack, are the hosts of this The New Stack Makers podcast. The featured guests are Bryan Liles, principal engineer, VMware and Cheryl Hung, vice president of ecosystem, CNCF.

In an era marked by complexity, the golden path is essential for software architects, asserts James Watters, senior director of R&D at VMware Tanzu, Broadcom. This approach, emphasizing fewer application patterns, simplifies life for security personnel, developers, and infrastructure teams. VMware defines the golden path as streamlining software development, crucial in today's economic climate. Watters highlights this in the Broadcom report: State of Cloud Native App Platforms 2024, noting that 55% of organizations favor this method for its consistency and security. 

Watters, a pioneer in platform as a service since 2009, helped establish Cloud Foundry and now drives VMware Tanzu. Tanzu's golden operations offer standardized, consistent processes across platforms, crucial for efficiency and security. Watters advocates for minimal DIY in favor of operational consistency, providing commands for building, deploying, and scaling applications. 

Tanzu’s focus is on integrating AI to enhance user interfaces and data access, impacting platform engineering significantly in the coming years. This integration aims to offer a better developer experience while maintaining security and efficiency. 

Learn more from The New Stack about golden paths: 

Golden Paths Start with a Shift Left 

Platform Engineering Not Working Out? You’re Doing It Wrong. 

How to Pave Golden Paths That Actually Go Somewhere 

Join our community of newsletter subscribers to stay on top of the news and at the top of your game. 

In this latest podcast from The New Stack, we spoke with Ricardo Torres, who is the chief engineer of open source and cloud native for aerospace giant Boeing. Torres also joined the Cloud Native Computing Foundation in May to serve as a board member. In this interview, recorded at KubeCon+CloudNativeCon last month, Torres speaks about Boeing's use of open source software, as well as its adoption of cloud native technologies.

While we may think of Boeing as an airplane manufacturer, it would be more accurate to think of the company as a large-scale system integrator, one that uses a lot of software. So, like other large-scale companies, Boeing sees a distinct advantage in maintaining good relations with the open source community.

"Being able to leverage the best technologists out there in the rest of the world is of great value to us strategically," Torres said. This strategy allows Boeing to "differentiate on what we do as our core business rather than having to reinvent the wheel all the time on all of the technology."

Like many other large companies, Boeing has created an open source office to better work with the open source community. Although Boeing is primarily a consumer of open source software, it still wants to work with the community. "We want to make sure that we have a strategy around how we contribute back to the open source community, and then leverage those learnings for inner sourcing," he said.

Boeing also manages how it uses open source internally, keeping tight controls on the supply chain of open source software it uses. "As part of the software engineering organization, we partner with our internal IT organization, to look at our internet traffic and assure nobody's going out and downloading directly from an untrusted repository or registry. And then we host instead, we have approved sources internally."

It's not surprising that Boeing, which deals with a lot of government agencies, embraces the practice of using software bills of material (SBOMs), which provide a full listing of what components are being used in a software system. In fact, the company has been working to extend the comprehensiveness of SBOMs, according to Torres.

" I think one of the interesting things now is the automation," he said of SBOMs. "And so we're always looking to beef up the heuristics because a lot of the tools are relatively naïve, and that they trust that the dependencies that are specified are actually representative of everything that's delivered. And that's not good enough for a company like Boeing. We have to be absolutely certain that what's there is exactly what did we expected to be there."

Cloud Native Computing

While Boeing builds many systems that reside in private data centers, the company is also increasingly relying on the cloud as well. Earlier this year, Boeing had signed agreements with the three largest cloud service providers (CSPs): Amazon Web Services, Microsoft Azure and the Google Cloud Platform.

"A lot of our cloud presence is about our development environments. And so, you know, we have cloud-based software factories that are using a number of CNCF and CNCF-adjacent technologies to enable our developers to move fast," Torres said.

Digital.ai, an AI-powered DevSecOps platform, serves large enterprises such as financial institutions, insurance companies, and gaming firms. The primary challenge faced by these clients is scaling their DevOps practices across vast organizations. They aim to combine modern development methodologies like agile DevOps with the need for speed and intimacy with end-users on a large scale.

This episode features a discussion between Wing To of Digital.ai and TNS host Heather Joslyn about platform engineering and the role of AI in enhancing automation. It delves into the dilemma of whether increased code production and release frequency driven by DevOps practices are inherently beneficial. Additionally, it explores the emerging challenge of AI-assisted development and how large enterprises are striving to realize productivity gains across their organizations.

Digital.ai is focused on incorporating AI into automation to assist developers in creating and delivering code while helping organizations derive more business value from their software in production. The company employs templates to capture and replicate key aspects of software delivery processes and uses AI to automate the rapid setup of developer environments and tooling. These efforts contribute to the concept of the internal developer platform, which consists of multiple toolsets for tasks like creating pipelines and setting up various components.

Learn more from The New Stack about Platform Engineering, DevSecOps and Digital.ai:

Platform Engineering Overview, News, and Trends

SRE vs. DevOps vs. Platform Engineering

Meet the New DevSecOps

Crash-level bugs continue to pose a significant challenge due to the lack of memory safety in programming languages, an issue persisting since the punch card era. This enduring problem, described as "the Joker to the Batman" by Anil Dash, VP of developer experience at Fastly, is highlighted in a recent episode of The New Stack Makers. The White House has emphasized memory safety, advocating for the adoption of memory-safe programming languages and better software measurability. The Office of the National Cyber Director (ONCD) noted that languages like C and C++ lack memory safety traits and are prevalent in critical systems. They recommend using memory-safe languages, such as Java, C#, and Rust, to develop secure software. Memory safety is particularly crucial for the US government due to the high stakes, especially in space exploration, where reliability standards are exceptionally stringent. Dash underscores the importance of resilience and predictability in missions that may outlast their creators, necessitating rigorous memory safety practices.

Learn more from The New Stack about Memory Safety:

White House Warns Against Using Memory-Unsafe Languages 

Can C++ Be Saved? Bjarne Stroupstrup on Ensuring Memory Safety

Bjarne Stroupstrup's Plan for Bringing Safety to C++

Join our community of newsletter subscribers to stay on top of the news and at the top of your game. 

In this episode of The New Stack Makers, Purnima Padmanabhan, a senior vice president at VMware, discusses three common mistakes organizations make when trying to move faster in meeting customer needs. The first mistake is equating application modernization with solely moving to the cloud, often resulting in a mere lift and shift of applications, without reaping the full benefits. The second mistake is a lack of automation, particularly in operations, which hinders the development process's speed. The third mistake involves adding unnecessary complexity by adopting new technologies or procedures, which slows down developers.

As a solution, Padmanabhan introduces the concept of platform engineering, which not only accelerates development but also reduces toil for operations engineers and architects. However, many organizations struggle with implementing it effectively, as they often approach platform engineering in fragmented ways, investing in separate components without fully connecting them.

To succeed in adopting platform engineering, Padmanabhan emphasizes the need for a mindset shift. The platform team must treat platform engineering as a continuously evolving product rather than a one-time delivery, ensuring that service-level agreements are continuously met, and regularly updating and improving features and velocity. The episode discusses the benefits of a well-implemented "golden path" for entire organizations and provides insights on how to start a platform engineering team.

Learn more from The New Stack about Platform Engineering and VMware:

Platform Engineering Overview, News and Trends

Platform Engineers: Developers Are Your Customers

Open Source Platform Engineering: A Decade of Cloud Foundry

Valkey, a Redis fork supported by the Linux Foundation, challenges Redis' new license. In this episode, Madelyn Olson, a lead contributor to the Valkey project and former Redis core contributor, along with Ping Xie, Staff Software Engineer at Google and Dmitry Polyakovsky, Consulting Member of Technical Staff at Oracle highlights concerns about the shift to a more restrictive license at Open Source Summit 2024 in Seattle. 

Despite Redis' free license for end users, many contributors may not support it. Valkey, with significant industry backing, prioritizes continuity and a smooth transition for Redis users. AWS, along with Google and Oracle maintainers, emphasizes the importance of open, permissive licenses for large tech companies. Valkey plans incremental updates and module development in Rust to enhance functionality and attract more engineers. The focus remains on compatibility, continuity, and consolidating client behaviors for a robust ecosystem. 

Learn more from The New Stack about the Valkey Project and changes to Open Source licensing

Linux Foundation Backs 'Valkey' Open Source Fork of Redis 

Redis Pulls Back on Open Source Licensing, Citing Stingy Cloud Services

HashiCorp's Licensing Change is only the Latest Challenge to Open Source 

Join our community of newsletter subscribers to stay on top of the news and at the top of your game. 

Brian Douglas was “the Beyoncé of GitHub.” He jokingly crowned himself with that title during his years at that company, where he advocated for open source and a more inclusive community supporting it. His work there eventually led to his new startup, Open Sauced.

Like the Queen Bey, Douglas’ mission is to empower a community. In his case, he’s seeking to support the open source community. With his former employer, GitHub, serving 4 million developers worldwide, the potential size of that audience is huge.

In this episode of The Tech Founder Odyssey podcast, he shared why empowerment and breaking down barriers to make anyone “awesome” in open source was the motivation behind his startup journey.

Beyoncé “has a superfan group, the Beyhive, that will go to bat for her,” Douglas pointed out. “So if Beyoncé makes a country song, the Beyhive is there supporting her country song. If she starts doing the house music, which is her latest album, [they] are there to the point where like, you cannot say bad stuff about, he pointed out,. So what I’m focused on is having a strong community and having strong ties.”

Open Sauced, which launched in June, seeks to build open source intelligence platform to help companies to stay competitive. Its aim is to help give more potential open source contributors the information they need to get started with projects, and help maintain them over time

The conversation was co-hosted by Colleen Coll and Heather Joslyn of The New Stack.

Web 2.0 ‘Opened the World’

Douglas’ introduction to tech started as a kid “cutting his teeth” on a Packard Bell and a shared computer at the community center inside his apartment complex, where he grew up outside of Tampa, Florida.

“I don't know what computer was in there, but it ran DOS,” he said. “And I got to play, like, Wolfenstein and eventually Duke Nukem and stuff like that. So that was my first sort of like, touch of a computer and I actually knew what I was doing.”

With his MBA in finance, the last recession in 2008 left only sales jobs available. But Douglas always knew he wanted to “build stuff.”

“I've always been like a copy and paste [person] and loved playing DOS games,” he told The New Stack. “I eventually [created] a pretty nice MySpace profile. then someone told me ‘Hey, you know, you could actually build apps now.’

“And post Web 2.0. people have frameworks and rails and Django. You just have to run a couple scripts, and you've got a web page live and put that in Heroku, or another server, and you're good. And that opened the world.”

Open Sauced began as a side project when he was director of developer advocacy at GitHub; He started working on the project full time in June, after about two years of tinkering with it.

Douglas didn’t grow up with money, he said, so moving from as an employee to the risky life of a CEO seeking funding prompted him to create his own comprehensive strategy. This included content creation (including a podcast, The Secret Sauce), other marketing, and shipping frontend code.

GitHub was very supportive of him spinning off Open Sauced as an independent startup, with colleagues assisting in refining his pitches to venture capital investors to raise funds.

“At GitHub, they have inside of their employee employment contract a moonlight clause,” Douglas said. Which means, he noted, because the company is powered by open source, “basically, whatever you work on, as long as you're not competing directly against GitHub, rebuilding it from the ground up, feel free to do whatever you need to do moonlight.”

Support for Blacks in Tech

Open Sauced will also continue Douglas’ efforts to increase representation of Blacks in tech and open pathways to level up their skills, similar to his work at GitHub with the Employee Resource Group (ERG) the Blacktocats.

“The focus there was to make sure that people had a home, like a community of belonging,” he said. “If you're a black employee at GitHub, you have a space and it was very helpful with things like 2020, during George Floyd. lt was the community [in which] we all supported each other during that situation.”

Douglas’ mission to rid the effects of imposter syndrome and champion anyone interested in open source makes him sound more like an open source ”whisperer”’ than a Beyoncé. Whatever the title, his iconic pizza brand — the company’s web address is “opensauced.pizza” — was his version, he said, of creating album cover art before forming the band.

His podcast’s tagline urges listeners to “stay saucy.” His plan for doing that at Open Sauced is to encourage new open source contributors.

“It's nice to know that projects can now opt in … but as a first-time contributor, where do I start? We can show you, ‘Hey, this project had five contributions, they're doing a great job. Why don't you start here?’

Sargun Kaur, co-founder of Byteboard, aims to revolutionize the tech interview process, which she believes is flawed and ineffective. In an interview with The New Stack for our Tech Founder Odyssey podcast series, Kaur compared assessing technical skills during interviews to evaluating the abilities of basketball star Steph Curry by asking him to draw plays on a whiteboard instead of watching him perform on the court. Kaur, a former employee of Symantec and Google, became motivated to change the interview process after a talented engineer she had coached failed a Google interview due to its impractical format.

Kaur believes that traditional tech interviews overly emphasize theoretical questions that do not reflect real-world software engineering tasks. This not only limits the talent pool but also leads to mis-hires, where approximately one in four new employees is unsuitable for their roles or teams. To address these issues, Kaur co-founded Byteboard in 2018 with Nicole Hardson-Hurley, another former Google employee. Byteboard offers project-based technical interviews, adopted by companies like Dropbox, Lyft, and Robinhood, to enhance the efficiency and fairness of their hiring processes. In recognition of their work, Kaur and Hardson-Hurley received Forbes magazine's "30 Under 30" award for enterprise technology.

Kaur's journey into the tech industry was unexpected, considering her initial disinterest in her father's software engineering career. However, exposure to programming and shadowing a female engineer at Microsoft sparked her curiosity, leading her to study computer science at the University of California, Berkeley. Overcoming initial challenges as a minority in the field, Kaur eventually joined Google as an engineer, content with the work environment and mentorship she received. However, her dissatisfaction with the interview process prompted her to apply to Google's Area 120 project incubator, leading to the creation of Byteboard. Kaur's experience with Byteboard's development and growth taught her valuable lessons about entrepreneurship, the power of founders in fundraising meetings, and the potential impact of AI on tech hiring processes.

Check out more episodes in The Tech Founder Odyssey series:

A Lifelong ‘Maker’ Tackles a Developer Onboarding Problem

How Teleport’s Leader Transitioned from Engineer to CEO

How 2 Founders Sold Their Startup to Aqua Security in a Year

Platform engineering is gaining prominence due to the need for faster application deployment, which directly impacts business velocity. Valentina Alaria, Senior Director of Product at VMware, emphasizes that not all organizations pursuing platform engineering have the same goals, context, or pain points. They tailor solutions to each organization's specific needs. Some focus on rapid onboarding for junior developers, while others aim to reduce complexity, friction, and support larger development teams with fewer operational staff.

Platform engineering aims to streamline collaboration between developers and operations engineers. Developers want portable code and the ability to focus on coding without worrying about production requirements. Operations engineers and platform teams seek a seamless environment for deploying applications in different contexts.

Successful platform engineering initiatives involve strong collaboration models, fostering a cooperative approach rather than a siloed one. The goal is to create applications and value for the organization by facilitating effective interaction between developers and operations engineers.

This podcast episode, hosted by Alex Williams of TNS, also delves into VMware Tanzu's latest tools for supporting platform engineering.

Learn more from The New Stack about platform engineering and VMware Tanzu:

Platform Engineering Overview, News and Trends

6 Patterns for Platform Engineering Success

A Guide to Open Source Platform Engineering

Streamline Platform Engineering with Kubernetes

OpenTelemetry is defined by its creators as a collection of APIs used to instrument, generate, collect and export telemetry data for observability. This data is in the form of metrics, logs and traces and has emerged as a popular CNCF project. For this interview, we're delving deeper into OpenTelemetry and its metrics support which has just become generally available.  

The specifications provided for the metrics protocol are designed to connect metrics to other signals and to provide a path to OpenCensus, which enables customers to migrate to OpenTelemetry and to work with existing metrics-instrumentation protocols and standards, including, of course, Prometheus. 

In this episode of The New Stack Makers podcast, recorded on the show floor of KubeCon + CloudNativeCon Europe 2022 in Valencia, Spain, Morgan McLean, director of product management, Splunk, Ted Young, director of developer education, LightStep and Daniel Dyla, senior open source architect, Dynatrace discussed how OpenTelemetry is evolving and the magic of observability in general for DevOps.

In March, Daniel Prizmant, senior security researcher for Palo Alto Networks, uncovered the malware targeting Windows containers, calling the exploit “Siloscape.” In a blog post, he wrote the emergence of such an attack was not “not surprising given the massive surge in cloud adoption over the past few years.”

In this edition of The New Stack Makers podcast, Prizmant, as the guest, described what makes Siloscape a threat for Kubernetes clusters — both for Linux and Windows containers.

The New Stack’s publisher and founder, Alex Williams, hosted this episode.

The internet's fabled history includes such milestones as the Advanced Research Projects Agency's (ARPA) development of packet switching (ARPANET), paving the way for today's modern infrastructure, or Tim Berners-Lee’s research that culminated in the explosive adoption of the World Wide Web (WEB) in the 1990s.  Today, as microservices, Kubernetes and distributed environments and connections become more prevalent, the use of the Internet is becoming more decentralized as well.

In this episode of The New Stack Makers podcast hosted by Alex Williams, founder and publisher of TNS, Storj Labs' Ben Golub, chairman and interim CEO, and Katherine Johnson, head of compliance, discuss how the Internet today centers around decentralization — and more importantly — how decentralization reflects upon the roots of the internet.

LOS ANGELES — When you’re deploying a business-critical application to the cloud, it’s nice to not need the “war room” you’ve assembled to troubleshoot Day 1 problems.

When BOK Financial, a financial services company that’s been moving apps to the cloud over the last three years, was launching its largest application on the cloud, its engineers supported it with a “war room type situation, monitoring everything” according to BOK’s Andrew Rau.

“After the first day, the system just scaled like it was supposed to … and they're like, ‘OK, I guess we don't need this anymore.’”

In this On the Road episode of The New Stack’s Makers podcast, Rau, BOK’s vice president and manager, cloud services, offered a case study about his organization’s cloud journey over the past four years, and the role HashiCorp’s Vault and Cloud Platform played in it.

Rau spoke to Heather Joslyn, features editor of The New Stack, about the challenges of moving a very traditional organization in a highly regulated industry to the cloud while maintaining tight security and resilience.

This episode of Makers, recorded in October at HashiConf in Los Angeles, was and sponsored by HashiCorp.

Upskilling for ‘Everything as Code’

In late 2019, Rau said, BOK Financial deployed one small application to the cloud, an initial step on its digital transformation journey. It’s been building out its cloud infrastructure ever since, and soon ran into the limits of each cloud provider’s native tooling.

“Where we struggled was we didn't want to deploy and manage our clouds in different ways,” he said. “We didn't want our cloud engineers to know just one cloud provider, and their technology and their tech stack. So that's when we really started looking at how else can we do this. And that's when Terraform was a great option for us.”

In 2020, BOK Financial began using HashCorp’s open source Terraform to automate the creation of cloud infrastructure. “We made a conscious effort to really focus on automation,” Rau said. “We didn't want to do things manually, which is really that traditional data center, how we've done things for decades.

In tandem with adopting Terraform, BOK Financial’s teams began using GitOps processes for CI/CD. But doing “everything as code,” as Rau put it, “required a lot of upskilling for some of our staff, because they've never done version control or automation capabilities. So in addition to learning Terraform, and these other cloud concepts, they had to learn all of that.”

The challenge, though, has been worth it: “It's really empowered us to move a lot faster, and give our application teams the ability to deploy at their pace, versus waiting on other teams.”

Seeking Automated Security

It took about a year, Rau said, to get BOK Financial’s developers comfortable using Terraform, largely because many were new to version control procedures and strategies.

Because the company works in a highly regulated industry, handling customers’ financial data, security is of utmost importance.

“We had users credentials for our clouds, and we had them separated out based on the type of deployment that [developers] were doing,” said Rau.

“But it wasn't easy for us to rotate those credentials on a frequent basis. And so we really felt the need that we want to make these short, limited tokens, no more than an hour for that deployment. And so that's where we looked at Vault.”

HashiCorp’s secrets storage and management tool proved an easy add-on with Terraform. “That's really given us the ability to have effectively no credentials — long-lived credentials — out there,” Rau said. “And secure our environment even more.” And because BOK’s teams don’t want to manage Vault and its complexities themselves, it has opted for HashiCorp Cloud Platform to manage it.

For other organizations on a cloud native journey, Rau recommended taking time to do things right. “We went back to rework some things periodically, because we learned something too late,” he said.

Also, he advised, keep stakeholders in the loop:  “You need to stay in front of the communication with business partners, IT leaders, that it's going to take longer to set this up. But once you do, it's incredible.”

Check out the podcast to learn more about BOK Financial's cloud native transformation.

When it comes to at-scale software development, is continuous delivery and release automation (CDRA) the next step in the evolution of continuous integration/continuous delivery (CI/CD)?

Forrester Research thinks so. The analysis firm describes CDRA as a way for organizations to deliver better-quality software faster and more securely, by automating digital pipelines and improving end-to-end management and visibility.

In this edition of The New Stack Makers podcast,  Anders Wallgren, CloudBees vice president of technology strategy, discusses CDRA, supporting tools and the goals and challenges DevOps teams have when delivering software today. CI/CD systems provider CloudBees was named a leading CDRA vendor in the report "The Forrester Wave: Continuous Delivery And Release Automation, Q2 2020."

The episode was hosted by Alex Williams, founder and publisher of The New Stack, and co-hosted by Joab Jackson, TNS managing editor.

Last week, the country of Spain dropped its mandate for residents and visitors to wear masks, to ward off further infections of the Coronavirus. So, for this year's KubeCon + CloudNativeCon  Europe conference, to be held May 16 - 20th of May in Valencia, Spain, the Cloud Native Computing Foundation dropped its own original mandate that attendees wear masks, a rule that had been in place for its other recent conferences.

This turned out to be the wrong decision, CNCF admitted a week later. A lot of people who already bought tickets were upset at this laxing of the rules for the conference, which could put them in greater danger of contacting the disease.

So the CNCF put the mandate back in place, and offered refunds for those who felt Spain's own decision would put them in harm's way. CNCF will even send you a week's worth of N95 masks if you request them.

So, long story short: bring a mask to KubeCon. And, as always, it is still a requirement to show proof of vaccination and temperature checks will be made as well.

Tricky business running a conference in this time, no?

In this latest episode of The New Stack Makers podcast, we take a look at what to expect from this year's KubeCon EU 2022. Our guests for this podcast are Priyanka Sharma, the executive director of CNCF, and Ricardo Rocha, who is a KubeCon co-chair and computer engineer at CERN. TNS Editor-in-chief Joab Jackson hosted this podcast.

We recorded this podcast prior to the discussion around masks, and at the time, Sharma said that the CNCF based the mask ruling on Spain's own country-wide mandates. "So we are being very cautious with the health requirements for the event," she said.

The conference team is also keeping an eye on Russia's aggressive moves in the Ukraine, though it is unlikely that the chaos will reach all the way to Spain. Still, "this is why it's essential to always have the hybrid option .. [to] have the virtual elements sorted," Sharma said.

As the CNCF flagship conference, KubeCon brings together managers and users of a wide variety of cloud native technologies, including containerd, CoreDNS, Envoy, etcd, Fluentd, Harbor, Helm, Istio, Jaeger, Kubernetes, Linkerd, Open Policy Agent, Prometheus, Rook, Vitess, Argo, CRI-O, Crossplane, dapr, Dragonfly,  Falco, Flagger, Flux, gRPC, KEDA, SPIFFE, SPIRE, and Thanos, and many many more. Most have been featured on TNS at one time or another.

In this podcast, we also discuss what to expect from the virtual sessions at the conference, what to do in Valencia, the current state of Kubernetes, and we get some unofficial picks from Sharma and Rocha as to what keynotes not miss and what sessions to attend.

"The virtual option is great," Rocha said. "But I think the in-person conferences have have their own value. And there's a lot to be to be gained about meeting people directly and exchanging ideas and going to these events on the side of the conference as well."

DETROIT — Modern software projects’ emphasis on agility and building community has caused a lot of security best practices, developed in the early days of the Linux kernel, to fall by the wayside, according to Aeva Black, an open source veteran of 25 years.

“And now we're playing catch up,“ said Black, an open source hacker in Microsoft Azure’s Office of the CTO  “A lot of less than ideal practices have taken root in the past five years. We're trying to help educate everybody now.”

Chris Short, senior developer advocate with Amazon Web Services (AWS), challenged the notion of “shifting left” and giving developers greater responsibility for security. “If security is everybody's job, it's nobody's job,” said Short, founder of the DevOps-ish newsletter.

“We've gone through this evolution: just develop secure code, and you'll be fine,” he said. “There's no such thing as secure code. There are errors in the underlying languages sometimes …. There's no such thing as secure software. So you have to mitigate and then be ready to defend against coming vulnerabilities.”

Black and Short talked about the state of the software supply chain’s security in an On the Road episode of The New Stack Makers podcast.

Their conversation with Heather Joslyn, features editor of TNS, was recorded at KubeCon + CloudNativeCon North America here in the Motor City.

This podcast episode was sponsored by AWS.

‘Trust, but Verify’

For our podcast guests, “trust, but verify” is a slogan more organizations need to live by.

A lot of the security problems that plague the software supply chain, Black said, are companies — especially smaller organizations — “just pulling software directly from upstream. They trust a build someone's published, they don't verify, they don't check the hash, they don't check a signature, they just download a Docker image or binary from somewhere and run it in production.”

That practice, Black said, “exposes them to anything that's changed upstream. If upstream has a bug or a network error in that repository, then they can't update as well.” Organizations, they said, should maintain an internal staging environment where they can verify code retrieved from upstream before pushing it to production — or rebuild it, in case a vulnerability is found, and push it back upstream.

That build environment should also be firewalled, Short added: “Create those safeguards of, ‘Oh, you want to pull a package from not an approved source or not a trusted source? Sorry, not gonna happen.’”

Being able to rebuild code that has vulnerabilities to make it more secure — or even being able to identify what’s wrong, and quickly — are skills that not enough developers have, the podcast guests noted.

More automation is part of the solution, Short said. But, he added, by itself it's not enough. “Continuous learning is what we do here as a job," he said. "If you're kind of like, this is my skill set, this is my toolbox and I'm not willing to grow past that, you’re setting yourself up for failure, right? So you have to be able to say, almost at a moment's notice, ‘I need to change something across my entire environment. How do I do that?’”

GitBOM and the ‘Signal-to-Noise Ratio’

As both Black and Short said during our conversation, there’s no such thing as perfectly secure code. And even such highly touted tools as software bills of materials, or SBOMs, fall short of giving teams all the information they need to determine code’s safety.

“Many projects have dependencies 10, 20 30 layers deep,” Black said. “And so if your SBOM only goes one or two layers, you just don't have enough information to know if as a vulnerability five or 10 layers down.”

Short brought up another issue with SBOMs: “There's nothing you can act on. The biggest thing for Ops teams or security teams is actionable information.”

While Short applauded recent efforts to improve user education, he said he’s pessimistic about the state of cybersecurity: “There’s not a lot right now that's getting people actionable data. It's a lot of noise still, and we need to refine these systems well enough to know that, like, just because I have Bash doesn't necessarily mean I have every vulnerability in Bash.”

One project aimed at addressing the situation is GitBOM, a new open source initiative. “Fundamentally, I think it’s the best bet we have to provide really high fidelity signal to defense teams,” said Black, who has worked on the project and produced a white paper on it this past January.

GitBOM — the name will likely be changed, Black said —takes the underlying technology that Git relies on, using a hash table to track changes in a project's code over time, and reapplies it to track the supply chain of software. The technology is used to build a hash table connecting all of the dependencies in a project and building what GItBOM’s creators call an artifact dependency graph.

“We had a team working on it a couple of proof of concepts right now,” Black said. “And the main effect I'm hoping to achieve from this is a small change in every language and compiler … then we can get traceability across the whole supply chain.”

In the meantime, Short said, there’s plenty of room for broader adoption of the best practtices that currently exist. “Security vendors, I feel,  like need to do a better job of moving teams in the right direction as far as action,” he said.

At DevOps Chicago this fall, Short said, he ran an open space session in which he asked participants for their pain points related to working with containers

“And the whole room admitted to not using least privilege, not using policy engines that are available in the Kubernetes space,” he said. “So there's a lot of complexity that we’ve got to help people understand the need for it, and how to implement it.”

Listen to whole podcast to learn more about the state of software supply chain security.

As the internet fills every nook and cranny of our lives, it runs into greater complexity for developers, operations engineers, and the organizations that employ them. How do you reduce latency? How do you comply with the regulations of each region or country where you have a virtual presence? How do you keep data near where it’s actually used?

For a growing number of organizations, the answer is to use the edge.

In this episode of Makers, the New Stack podcast, Ron Lev, general manager of Cox Edge,  and Sheraline Barthelmy, head of product,  marketing and customer success for Cox Edge, were joined by Chetan Venkatesh, founder and CEO of Macrometa. The trio discussed the best use cases for edge computing, the advantages it can bring, and the challenges that remain.

The podcast was hosted by Heather Joslyn, features editor of The New Stack.

In deploying cloud-native sustainable foundation AI models, there are five key steps outlined by Huamin Chen, an R&D professional at Red Hat's Office of the CTO. The first two steps involve using containers and Kubernetes to manage workloads and deploy them across a distributed infrastructure. Chen suggests employing PyTorch for programming and Jupyter Notebooks for debugging and evaluation, with Docker community files proving effective for containerizing workloads.

The third step focuses on measurement and highlights the use of Prometheus, an open-source tool for event monitoring and alerting. Prometheus enables developers to gather metrics and analyze the correlation between foundation models and runtime environments.

Analytics, the fourth step, involves leveraging existing analytics while establishing guidelines and benchmarks to assess energy usage and performance metrics. Chen emphasizes the need to challenge assumptions regarding energy consumption and model performance.

Finally, the fifth step entails taking action based on the insights gained from analytics. By optimizing energy profiles for foundation models, the goal is to achieve greater energy efficiency, benefitting the community, society, and the environment.

Chen underscores the significance of this optimization for a more sustainable future.

Learn more at thenewstack.io

PyTorch Takes AI/ML Back to Its Research, Open Source Roots

PyTorch Lightning and the Future of Open Source AI

Jupyter Notebooks: The Web-Based Dev Tool You've Been Seeking

Know the Hidden Costs of DIY Prometheus

The  number of open source components inside services and applications continues to increase exponentially, and this adoption is creating a lot of change in how software is created, deployed and managed.  in 2016, applications on average had 86 open source software components. Today, the average number of components is 528, according to “The 2021 Open Source Security and Risk Analysis (OSSRA) report.”

In this latest edition of The New Stack Makers podcast, we discuss the implications of the explosion of  open source’s adoption and its effect on data center operations.

The guests were Mark Hinkle, co-founder and CEO, TriggerMesh, Shaun O’Meara, field CTO, Mirantis; Jeremy Tanner, developer relations, Equinix and Sophia Vargas, research analyst, open source programs office, Google.

TNS’ Founder and Publisher Alex Williams and TNS Editor Joab Jackson hosted this podcast.

VALENCIA, SPAIN —Managing the cloud virtual machines (VMs) your containers run on. Running data-intensive workloads. Scaling services in response to spikes in traffic — but doing so in a way that doesn’t jack up your organization’s cloud spend. Kubernetes (K8s) seems so easy at the beginning, but it brings challenges that rachet up complexity as you go.

The cloud native ecosystem is filling up with tools aimed at making these challenges easier on developers, data scientists and Ops engineers. Increasingly, automation is the secret sauce helping teams and their companies work faster, safer and more productively.

In this special On the Road edition of The New Stack Makers podcast recorded at [sponsor_inline_mention slug="kubecon-cloudnativecon" ]KubeCon + CloudNativeCon EU[/sponsor_inline_mention], we unpacked some of the ways automation helps simplify Kubernetes. We were joined by a trio of guests from [sponsor_inline_mention slug="netapp" ]Spot.io by NetApp[/sponsor_inline_mention]: Jean-Yves “JY” Stephan, senior product manager  for Ocean for Apache Spark, along with Gilad Shahar, ​and Yarin Pinyan —product manager and  product architect, respectively, for Spot.io.

Until recently, Stephan noted, Apache Spark, the open source, unified analytics engine for large-scale data processing, couldn’t be deployed on K8s. “So all these regular software engineers were getting the cool technology with Kubernetes, cloud native solutions,” he said. “And the big data engineers, they were stuck with technologies from 10 years ago.”

Spot.io, he said, lets Apache Spark run atop Kubernetes: “It’s a lot more developer friendly, it’s a lot more flexible and it can also be more cost effective.”

The company’s Ocean CD, expected to be generally available in August, is aimed at solving another Kubernetes problem, said Pinyan: canary deployments.

Previously, if you were running normal VMs, without Kubernetes, it was pretty easy to do canary deployments because you had to scale up a VM and then see if the new version worked fine on it, and then gradually scale the others,” he said. “In Kubernetes, it’s pretty complex, because you have to deal with many pods and deployments.”

In enterprises, where DevOps and SRE team members are likely serving multitudes of developers, automating as much toil as possible for devs is essential, said Shahar. For instance, Spot.io’s tools allow users to “break the configuration into parts,” he said, which can task developers with whatever percentage of responsibility for the config that is deemed best for their use case.

“We try to design our solutions in a way that will allow the DevOps [team] to set things once and basically provide pre-baked solutions for the developers,” he said. “Because the developer, at the end of the day, knows best what their application will require.”

From cloud security providers to open source, trust has become a staple from which an organization's security is built. But with the rise of cloud-native technologies, the new ways of building applications are challenging the traditional approaches to security. The changing cloud-native landscape is requiring broader security coverage across the technology stack and more contextual awareness of the environment. So how should DevOps and InfoSec teams across commercial businesses and governments rethink their security approach?

In this episode of The New Stack Makers podcast, Tom Bossert, president of Trinity Cyber (and former Homeland Security Advisor to two Presidents); Patrick Hylant, client executive of VMware; and Chenxi Wang, managing general partner, Rain Capital discuss how businesses and the U.S. government can adapt to the evolving threat landscape, including new initiatives and lessons that can be applied in this high-risk environment.

Alex Williams, founder and publisher of The New Stack, hosted this podcast. Jim Douglas, CEO of Armory also joined as co-host of this livestream event.

The practice of "shift left," which involves moving security concerns to the code level and increasing developers' responsibility for security, is facing a backlash, with both developers and security professionals expressing concerns. Peter Klimek, director of technology at Imperva, discusses the reasons behind this backlash in this episode.

Some organizations may have exhausted the benefits of shift left, while the main challenge for many isn't finding vulnerabilities but finding time to address them. Security attacks are now targeting business logic vulnerabilities rather than dependencies, which shift left tools are better at identifying. These business logic vulnerabilities are often tied to authorization decisions, making them harder to address through code-level tools. Additionally, attacks increasingly focus on the frontend, such as API development and cart attacks.

Klimek emphasizes the need for development and security teams to collaborate and advocates for using DORA metrics to assess the impact of security efforts on the development pipeline. Some organizations may reach a point where the tools added to the development lifecycle become counterproductive, he notes. DORA metrics can help determine when this occurs and provide valuable insights for security teams.

Learn more from The New Stack about Developer Security and Imperva:

Why Your APIs Aren’t Safe — and What to Do about It

What Developers Need to Know about Business Logic Attacks

Are Your Development Practices Introducing API Security Risks?

DETROIT — Developer relations, or DevRel to its friends, is not only a coveted career path but also essential to helping developers learn and adopt new technologies.

That guidance is a matter of survival for many organizations. The cloud native era demands new skills and new ways of thinking about developers and engineers’ day-to-day jobs. At Dell Technologies, it meant responding to the challenges faced by its existing customer base, which is “very Ops centric — server admins, system admins,” according to Brad Maltz, of Dell.

With the rise of the DevOps movement, “what we realized is our end users have been trying to figure out how to become infrastructure developers,” said Maltz, the company’s senior director of DevOps portfolio and DevRel. “They've been trying to figure out how to use infrastructure as code Kubernetes, cloud, all those things.”

“And what that means is we need to be able to speak to them where they want to go, when they want to become those developers. That’s led us to build out a developer relations program ... and in doing that, we need to grow out the community, and really help our end users get to where they want to.”

In this episode of The New Stack’s Makers podcast, Maltz spoke to Heather Joslyn, TNS features editor, about how Dell has, since August, been busy creating a DevRel team to aid its enterprise customers seeking to adopt DevOps as a way of doing business.

This On the Road edition of Makers, recorded at KubeCon + CloudNativeCon North America in the Motor City, was sponsored by Dell Technologies.

Recruiting Influencers

Maltz, an eight-year veteran of Dell, has moved quickly in assembling his team, with three hires made by late October and a fourth planned before year’s end. That’s lightning fast, especially for a large, established company like Dell, which was founded in 1984.

“There's two ways of building a DevOps team,” he said. “One way is to actually kind of go and try to homegrow people on the inside and get them more presence in the community. That's the slower road.

“But we decided we have to go and find industry influencers that believe in our cause, that believe in the problem space that we live in. And that's really how we started this: we went out to find some very, very strong top talent in the industry and bring them on board.”

In addition to spreading the DevOps solutions gospel at conferences like KubeCon, Maltz’s vision for the team is currently focused on social media and building out a website, developer.dell.com, which will serve as the landing page for the company’s DevRel knowledge, including links to community, training, how-to videos and an API marketplace.

In building the team, the company made an unorthodox choice. “We decided to put Dev Rel into product management on the product side, not marketing,” Maltz said. “The reason we did that was we want the DevRel folks to really focus on community contributions, education, all that stuff.

“But while they're doing that, their job is to bring the data back from those discussions they're having in the field back to product management, to enable our tooling to be able to satisfy some of those problems that they're bringing back so we can start going full circle.”

Facing the Limits of ‘Shift Left’

The roles that Dell’s DevRel team is focusing on in the DevOps culture are site reliability engineers (SREs) and platform engineers. These not only align with its traditional audience of Ops engineers, but reflect a reality Dell is seeing in the wider tech world.

“The reality is, application developers don't want to shift left, they don't want to operate. They don't want they want somebody else to take it, and they want to keep developing,” Maltz said.  “where DevOps has transitioned for us is, how do we help those people that are kind of that operator turning into infrastructure developer fit into that DevOps culture?”

The rise of platform engineering, he suggested, is a reaction to the endless choices of tools available to developers these days.

“The notion is developers in the wild are able to use any tool on any cloud with any language, and they can do whatever they want. That's hard to support,” he said.

“That's where DevOps got introduced, and was to basically say, Hey, we're gonna put you into a little bit of a box, just enough of a box that we can start to gain control and get ahead of the game. The platform engineering team, in this case, they're the ones in charge of that box.”

But all of that, Maltz said, doesn’t mean that “shift left” — giving devs greater responsibility for their applications — is dead. It simply means most organizations aren’t ready for it yet: “That will take a few more years of maturity within these DevOps operating models, and other things that are coming down the road.”

Check out the full episode for more from Maltz, including new solutions from Dell aimed at platform engineers and SREs and collaborations with Red Hat OpenShift.

The use of large language models (LLMs) has become widespread, but there are significant security risks associated with them. LLMs with millions or billions of parameters are complex and challenging to fully scrutinize, making them susceptible to exploitation by attackers who can find loopholes or vulnerabilities. On an episode of The New Stack Makers, Chris Pirillo, Tech Evangelist and Lance Seidman, Backend Engineer at Atomic Form discussed these security challenges, emphasizing the need for human oversight to protect AI systems.

One example highlighted was malicious AI models on Hugging Face, which exploited the Python pickle module to execute arbitrary commands on users' machines. To mitigate such risks, Hugging Face implemented security scanners to check every file for security threats. However, human vigilance remains crucial in identifying and addressing potential exploits.

Seidman also stressed the importance of technical safeguards and a culture of security awareness within the AI community. Developers should prioritize security throughout the development life cycle to stay ahead of evolving threats. Overall, the message is clear: while AI offers remarkable capabilities, it requires careful management and oversight to prevent misuse and protect against security breaches.

Learn more from The New Stack about AI and security:  

Artificial Intelligence: Stopping the Big Unknown in Application, Data Security 

Cyberattacks, AI and Multicloud Hit Cybersecurity in 2023

Will Generative AI Kill DevSecOps? 

Join our community of newsletter subscribers to stay on top of the news and at the top of your game.

Developer tool integration and AI differentiate workflows to achieve that "fluid" state developers strive for in their work.

Amazon CodeCatalyst and Amazon CodeWhisperer exemplify how developer workflows are accelerating and helping to create these fluid states. That's a big part of the story we hear from Harry Mower, director AWS DevOps Services, and Doug Seven, director, Software Development, AWS CodeWhisperer, from our recording in Seattle earlier in April for this week's  AWS Developer Innovation Day.

CodeCatalyst serves as an end-to-end integrated DevOps toolchain that provides developers with everything they need to go from planning through to deployment, Mower said. CodeWhisperer is an AI coding companion that generates whole-line and full-line function code recommendations in an integrated development environment (IDE).

CodeWhisperer is part of the IDE, Seven said. The acceleration is two-fold. CodeCatalyst speeds the end-to-end integration process, and CodeWhisper accelerates writing code through generative AI.

Software deployments increasingly involve highly distributed and decentralized application development processes for deployments across any combination of data centers, public cloud and to the edge. All the while, reliability, security or performance cannot be compromised.

In this The New Stack Makers podcast, a panel of technology executives discussed the best ways to speed up business innovation in today’s multicloud and multi-infrastructure world. They also discussed how to deliver apps and services faster to improve the customer experience — over a pancake breakfast during VMworld, VMware’s annual user’s conference.

The guests were Dormain Drewitz, senior director of product marketing for VMware Tanzu, Mandy Storbakken, cloud technologist for VMware, Shawn Bass, CTO for VMware’s end-user computing business, and Jo Peterson, vice president cloud and security services, Clarify360.

Alex Williams, founder and publisher of TNS, and Joab Jackson, TNS editor-in-chief, hosted the podcast.

Ask a developer about how they got into programming, and you learn so much about them.

In this week's episode of The New Stack Makers, Chainguard founder Dan Lorenc said he got into programming halfway through college while studying mechanical engineering.

"I got into programming because we had to do simulations and stuff in MATLAB," Lorenc said. And then I switched over to Python because it was similar. And we didn't need those licenses or whatever that we needed. And then I was like, Oh, this is much faster than you know, ordering parts and going to the machine shop and reserving time, so I got into it that way."

It was three or four years ago that Lorenc got into the field of open source security.

"Open source security and supply chain security weren't buzzwords back then," Lorenc said. "Nobody was talking about it. And I kind of got paranoid about it."

Lorenc worked on the Minikube open source project at Google where he first saw how insecure it could be to work on open source projects. In the interview, he talks about the threats he saw in that work.

It was so odd for Lorenc. State of art for open source security was not state of the art at all. It was the stone age.

Lorenc said it felt weird for him to build the first release in MiniKube that did not raise questions about security.

"But I mean, this is like a 200 megabyte Go binary that people were just running as root on their laptops across the Kubernetes community," Lorenc said. "And nobody had any idea what I put in there if it matched the source on GitHub or anything. So that was pretty terrifying. And that got me paranoid about the space and kind of went down this long rabbit hole that eventually resulted in starting Chainguard.

Today, the world is burning down, and that's good for a security startup like Chainguard.

"Yeah, we've got a mess of an industry to tackle here," Lorenc said. "If you've been following the news at all, it might seem like the software industry is burning on fire or falling down or anything because of all of these security problems. It's bad news for a lot of folks, but it's good news if you're in the security space."

Good news, yes ,but how does it fit into a larger story?

"Right now, one of our big focuses is figuring out how do we explain where we fit into the bigger landscape," Lorenc. said. "Because the security market is massive and confusing and full of vendors, putting buzzwords on their websites, like zero trust and stuff like that. And it's pretty easy to get lost in that mess. And so figuring out how we position ourselves, how we handle the branding, the marketing, and making it clear to prospective customers and community members, everything exactly what it is we do and what threats our products mitigate, to make sure we're being accurate there. And conveying that to our customers. That's my big focus right now."

Let’s say you’re a passenger on a cruise ship. Floating in the middle of the ocean, far from reliable Wi-Fi, you wear a device that lets you into your room, that discreetly tracks your move from the bar to the dinner table to the pool and delivers your drink order wherever you are. You can buy sunscreen or toothpaste or souvenirs in the ship’s stores without touching anything.

If you’re a Carnival Cruise Lines passenger, this is reality right now, in part because of the company’s partnership with Couchbase, according to Mark Gamble, product and solutions marketing director, Couchbase.

Couchbase provides a cloud native, no SQL database technology that's used to power applications for customers including Carnival but also Amadeus, Comcast, LinkedIn, and Tesco.

In Carnival’s case, Gamble said, “they run an edge data center on their ships to power their Ocean Medallion application, which they are super proud of. They use it a lot in their ads, because it provides a personalized service, which is a differentiator for them to their customers.”

In this episode of The New Stack Makers, Gamble spoke to Heather Joslyn, features editor of TNS, about edge computing, 5G, and Couchbase Capella, its Database as a Service (DBaaS) offering for enterprises.

This episode of Makers was sponsored by Couchbase.

5G and Offline-First Apps

The goal of edge computing, Gamble told our podcast audience, is bring data and compute closer to the applications that consume it. This speeds up data processing, he said, “because data doesn't have to travel all the way to the cloud and back.” But it also has other benefits

“This serves to make applications more reliable, because local data processing sort of removes internet slowness and outages from the equation,” he said.

The innovation of 5G networks has also had a big impact on reducing latency and increasing uptime, Gamble said.

“To compare with 4G, things like the average round trip data travel time between the device, and the cell tower is like 15 milliseconds. And with 5G, that latency drops to like two milliseconds. And 5G can support they say, a million devices, within a third of a mile radius, way more than what's possible with 4G.”

But 5G, Gamble said, “really requires edge computing to realize its its full potential.” Increasingly, he said, Couchbase hears interest from its customers in building “offline-first” applications, which can run even in Wi-Fi dead zones.

The use cases, he said, are everywhere: “When I pass a fast food restaurant, it's starting to become more common, where you'll see that, instead of just a box you're talking to, there's a person holding a tablet, and they walk down the line, and they're taking orders. And as they come closer to the restaurant, it syncs up with the kitchen. They find that just a better, more efficient way to serve customers. And so it becomes a competitive differentiator forum.”

As part of Couchbase’s Capella product, it recently announced Capella App Service, a new capability for mobile developers, is a fully managed backend designed for mobile, Internet of Things (IoT) and edge applications.

“Developers use it to access and sync data between the Database as a Service and their edge devices, as well as it handles authenticating and managing mobile and edge app users,” he said.

Used in conjunction with Couchbase Lite, a lightweight, embedded NoSQL database used with mobile and IoT devices, Capella App Services synchronizes the data between backend and edge devices.

Even for workers in remote areas, “eventually, you have to make sure that data updates are shared with the rest of the ecosystem,” Gamble said. “ And that's what App Services is meant to do, as conductivity allows — so during network disruptions in areas with no internet, apps will still continue to operate.”

Check out the rest of the conversation to learn more about edge computing and the challenges Gamble thinks still need to be addressed in that space.

At last count, social media giant Twitter enjoys around 353 million active users, and streaming music service Spotify has 356 million active listeners. In both cases, open source tools and platforms for cloud native environments have served as the cornerstones for their tremendous growth.

In this latest episode of The New Stack Makers podcast, Spotify Senior Staff Engineer Dave Zolotusky, and Twitter Developer Experience Lead and Manager for Engineering Effectiveness Jasmine James discussed the role of open source software in their respective organizations. Katie Gamanj, ecosystem manager of the Cloud Native Computing Foundation and Alex Williams, founder and publisher of TNS,  co-hosted this interview.

Once they have piloted Kubernetes, many organizations then want to scale up their K8s deployments, and run workloads across many clusters. But managing multiple clusters requires a new set of tools, ones that automate many routine and manual tasks. So, for its fifth Tech Radar report, the Cloud Native Computing Foundation surveyed the tools available for multicluster management, based on the input from its end-user community.

In this edition of The New Stack Analysts podcast, we talk with two people who helped assemble the report, Federico Hernandez, principal engineer social media analysis provider Meltwater, and Simone Sciarrati, Meltwater engineering team lead. We chatted about the report's findings and how the multicluster management tool landscape is taking shape. Co-hosting this episode is Alex Williams, founder and publisher of The New Stack and the Tech Radar's organizer Cheryl Hung, CNCF vice president of ecosystem.

In this episode of The New Stack’s On the Road show at Open Source Summit in Austin, Julia Ferraioli, open source technical leader at Cisco’s open source programs office, spoke with The New Stack about some alternative ways to define what is and is not ‘open source.’

When someone says, well, that’s ‘technically’ open source, it’s usually to be snarky about a project that meets the legal criteria to be open source, but doesn’t follow the spirit of open source. Ferraioli doesn’t think that the ‘classic’ open source project, like a Kubernetes or Linux, are the only valid models for open source. She gives the sample of a research project — the code might be open sourced specifically so that others can see the code and reproduce the results themselves. However, for the research to remain valid, they it can’t accept any contributions.

“It’s no less open source than others,” Ferraioli said about the hypothetical research project. “If you break things down by purpose, it’s not always that you’re trying to build the robust community.” The social model of open source, Ferraioli says, is about understanding the different use cases for open source, as well as providing a framework for determining what appropriate success metrics could be depending on what the project’s motivations are. And if you’re just doing a project with friends for laughs, well, quantifying fun isn’t going to be easy.

DUBLIN — The mission of Linux Foundation Energy —  a collaborative, international effort by power companies to help move the world away from fossil fuels — has never seemed more urgent.

In addition to the increased frequency and ferocity of extreme weather events like hurricanes and heat waves, the war between Russia and Ukraine has oil-dependent countries looking ahead to a winter of likely energy shortages.

“I think we need to go faster,” said Benoît Jeanson, an enterprise architect at RTE, the French electricity transmission system operator.  He aded, “What we are doing with the Linux Foundation Energy is really something that will help for the future, and we need to go faster and faster.

For this On the Road episode of The New Stack’s Makers podcast, recorded at Open Source Summit Europe here, we were joined by two guests who work in the power industry and whose organizations are part of LF Energy.

In addition to Jeanson, this episode featured Jonas van den Bogaard, a solution architect and open source ambassador at Alliander, an energy network company that provides energy transport and distribution to a large part of the Netherlands. Van den Bogaard also serves on the technical advisory council of LF Energy.

Heather Joslyn, features editor of TNS, hosted this conversation.

18 Open Source Projects

LF Energy, started in 2018, now includes 59 member organizations, including cloud providers Google and Microsoft, enterprises like General Electric, and research institutions like Stanford University. It currently hosts 18 open source projects; the podcast guests encouraged listeners to check them out and contribute to them.

Among them: OpenSTEF, automated machine learning pipelines to deliver accurate forecasts of the load on the energy grid 48 hours ahead of time. “It gives us the opportunity to take action in time to prevent the maximum grid capacity [from being] reached,” said van den Bogaard.

“That’s going to prevent blackouts and that sort of thing. And also, another side: it makes us able to add renewable energies to the grid.”

Jeanson said that the open source projects aim to cover “every level of the stack. We also have tools that we want to develop at the substation level, in the field.” Among them: OperatorFabric, Written in Java and based on the Spring framework, OperatorFabric is a modular, extensible  platform for systems operators, including several features aimed at helping utility operators.

It helps operators coordinate the many tasks and alerts they need to keep track of by aggregate notifications from several applications into a single screen.

“Energy is of importance for everyone,” said van den Bogaard. “And especially moving to more cleaner and renewable energy is key for us all. We have great minds all around the world. And I really believe that we can achieve that. The best way to do that is to combine the efforts of all those great minds. Open source can be a great enabler of that.”

Cultural Education Needed

But persuading decision-makers in the power industry to participate in building the next generation of open source solutions can be a challenge, van den Bogaard acknowledged.

“You see, that the energy domain has been there for a long time, and has been quite stable, up to like 10 years ago.” he said. In such a tradition-bound culture, change is hard. In the cloud era, he added, a lot of organizations “need to digitalize and focus more on it and those capabilities are new. And also, open source, for in that matter is also a very new concept.”

One obstacle in the energy industry taking more advantage of open source tools, Jeanson noted, is security: “Some organizations still see open source to be a potential risk.” Getting them on board, he said, requires education and training.

He added, “vendors need to understand that open source is an opportunity that they should not be afraid of. That we want to do business with them based on open source. We just need to accelerate the momentum.

Check out the whole episode to learn more about LF Energy’s work.

In this latest episode of The New Stack Makers podcast, we delve more deeply into the emerging practice of platform engineering. The guests for this show are Aeris Stewart, community manager at platform orchestration provider Humanitec and  Michael Galloway, an engineering leader for infrastructure software provider HashiCorp. TNS Features Editor Heather Joslyn hosted this conversation.

Although the term has been around for several years, platform engineering caught the industry's attention in a big way last September, when Humanitec published a report that identified how widespread the practice was quickly becoming, citing its use by Nike, Starbucks, GitHub and others.

Right after the report was released, Stewart provided an analysis for TNS arguing that platform engineering solved the many issues that another practice, DevOps, was struggling with. "Developers don’t want to do operations anymore, and that’s a bad sign for DevOps," Stewart wrote. The post stirred a great deal of conversation around the success of DevOps.

Platform engineering is "a discipline of designing and building tool chains and workflows that enable developer self service," Stewart explained. The purpose is to give the developers in your organization a set of standard tools that will allow them to do their job — write and fix apps — as quickly as possible. The platform provides the tools and services "that free up engineering time by reducing manual toil cognitive load," Galloway added.

But platform engineering also has an advantage for the business itself, Galloway elaborated. With an internal developer platform in place, a business can scale up with "reliability, cost efficiency and security," Galloway said.

Before HashiCorp, Galloway was an engineer at Netflix, and there he saw the benefits of platform engineering for both the dev and the business itself. "All teams were enabled to own the entire lifecycle from design to operation. This is really central to how Netflix was able to scale," Galloway said. A platform engineering team created a set of services that made it possible for Netflix engineers to deliver code "without needing to be continuous delivery experts."

The conversation also touched on the challenges of implementing platform engineering, and what metrics you should use to quantify its success.

And because platform engineering is a new discipline, we also discussed education and community. Humanitec's debut PlatformCon drew over 6,000 attendees last June (and Platform 2023 has just been scheduled for June).  There is also a platform engineering Slack channel, which has drawn over 8,000 participants thus far.

"I think the community is playing a really big role right now, especially as a lot of organizations' awareness of platform engineering is just starting," Stewart said. "There's a lot of knowledge that can be gained by building a platform that you don't necessarily want to learn the hard way."

When she was a student in her native Israel, Shira Shamban was a self-proclaimed “geek.”

But, unusually for a tech company founder and CEO, not a computer geek.

Shamban was a science nerd, with her sights set on becoming a doctor. But first, she had to do her state-mandated military service. And that’s where her path diverged.

In the military, she was not only immersed in computers but spent years working in intelligence; she stayed in the service for more than a decade, eventually rising to become head of an intelligence sector for the Israeli Defense Forces. At home, she began building her own projects to experiment with ideas that could help her team.

“So that kind of helped me not to be intimidated by technology, to learn that I can learn anything I want by myself,” said Shamban, co-founder of Solvo, a company focused on data and cloud infrastructure security. “And the most important thing is to just try out things that you learn.”

To date, Solvo has raised about $11 million through investors like Surround Ventures, Magenta Venture Partners, TLV Partners and others. In this episode of The New Stack Makers podcast series The Tech Founder Odyssey, Shamban talked to Heather Joslyn and Colleen Coll of TNS about her journey.

In-Person Teamwork

Shamban opted to stay in the technology world, nurturing a desire to eventually start her own company. It was during a stint at Dome9, a cloud security company, that she met her future Solvo co-founder, David Hendri — and built a foundation for entrepreneurship.

“After that episode, I got the guts,” she said. “Or I got stupid enough.”

Hendri, now Solvo’s chief technology officer, struck Shamban as having the right sensibility to be a partner in a startup. At Dome9, she said, “very often, I used to stay up late in the office, and I would see him as well. So we'd grab something to eat.”

Their casual conversations quickly revealed that Hendri was often staying late to troubleshoot issues that were not his or his team’s responsibility, but simply things that someone needed to fix. That sense of ownership, she realized, “is exactly the kind of approach one would need to bring to the table in a startup.”

The mealtime chats that started Solvo have carried over into its current organizational culture. The company employs 20 people; workers based in Tel Aviv are expected to come to the office four days a week.

Hendri and Shamban started their company in the auspicious month of March 2020, just as the Covid-19 pandemic started. While many companies have moved to all-remote work, Solvo never did.

“We knew we wanted to sit together in the same room, because the conversations you have over a cup of coffee are not the same ones that you have on a chat, and on Slack,” the CEO said. “So that was our decision. And for a long time, it was an unpopular decision.”

As the company scales, finding employees who align with its culture can make recruiting tricky, Shamban said.

It's not only about your technical expertise, it's also about what kind of person you are,” she said. “Sometimes we found very professional people that we didn't think would make a good fit to the culture that we want to build. So we did not hire them. And in the boom times, when it was really hard to hire engineers.

“These were tough decisions. But we had to make them because we knew that building a culture is easier in a way than fixing a culture.

Listen to the full episode to hear more about Shamban's journey.

Both APIs and microservices play a key role in cloud native environments. Microservices serve as the cornerstone of distributed and shared computing resources. At the same time, APIs serve as a very efficient way to streamline many operations and development tasks from DevOps teams.

However, both microservices and APIs carry with them their own security risks.  All it takes is for one compromised Kubernetes node to allow for an intruder to gain root access through an API to an organization’s entire container infrastructure across multiple clusters (a worst-case scenario).

In this episode of The New Stack Makers podcast, we look at how to both secure microservices with APIs and how to rely on APIs to delegate certain security tasks to a trusted third party. Our guest is Viktor Gamov, principal developer advocate for Kong, an API-connectivity company. The episode is hosted by Alex Williams, TNS founder and publisher, and Bharat Bhat, marketing lead, developer relations, Okta.

The most attractive characteristic of open-source projects is the potential to tap into the total addressable market of collaborators. But when looking for users to your project and building a community around it requires the project to stand out from the millions of others, how do you build a plan to monetize it?

In this podcast, Emily Omier, a positioning consultant who works with startups to stake out the right position in the cloud native / Kubernetes ecosystem, discusses how to grow your project by finding the right market category for your open-source startup.

Alex Williams, founder and publisher of The New Stack hosted this podcast.

Boeing, with around 6,000 engineers, is emphasizing open source engagement by focusing on three main themes, according to Damani Corbin, who heads Boeing's Open Source office. He joined our host, Alex Williams, for a discussion at KubeCon+CloudNativeCon in Chicago.

The first priority Corbin talks about is simplifying the consumption of open source software for developers. Second, Boeing aims to facilitate developer contributions to open source projects, fostering involvement in communities like the Cloud Native Computing Foundation and the Linux Foundation. The third theme involves identifying opportunities for "inner sourcing" to share internally developed solutions across different groups.

Boeing is actively working to break down barriers and encourage code reuse across the organization, promoting participation in open source initiatives. Corbin highlights the importance of separating business-critical components from those that can be shared with the community, prioritizing security and extending efforts to enhance open source security practices. The organization is consolidating its open source strategy by collaborating with legal and information security teams.

Corbin emphasizes the goal of making open source involvement accessible and attractive, with a phased approach to encourage meaningful contributions and ultimately enabling the compensation of engineers for open source work in the future.

Learn more from The New Stack about Boeing and CNCF open source projects:

How Boeing Uses Cloud Native

How Open Source Has Turned the Tables on Enterprise Software

Scaling Open Source Community by Getting Closer to Users

Mercedes-Benz: 4 Reasons to Sponsor Open Source Projects

The Linux xz utils backdoor exploit, discussed in an interview at the Open Source Summit 2024 on The New Stack Makers with John Kjell, director of open source at TestifySec, highlights critical vulnerabilities in the open-source ecosystem. This exploit involved a maintainer of the Linux xz utils project adding malicious code to a new release, discovered by a Microsoft engineer. This breach demonstrates the high trust placed in maintainers and how this trust can be exploited. Kjell explains that the backdoor allowed remote code execution or unauthorized server access through SSH connections.

The exploit reveals a significant flaw: the human element in open source. Maintainers, often under pressure from company executives to quickly address vulnerabilities and updates, can become targets for social engineering. Attackers built trust within the community by contributing to projects over time, eventually gaining maintainer status and inserting malicious code. This scenario underscores the economic pressures on open source, where maintainers work unpaid and face demands from large organizations, exposing the fragility of the open-source supply chain. Despite these challenges, the community's resilience is also evident in their rapid response to such threats.

Learn more from The New Stack about Linux xz utils 

Linux xz Backdoor Damage Could Be Greater Than Feared 

Unzipping the XZ Backdoor and Its Lessons for Open Source 

The Linux xz Backdoor Episode: An Open Source Myster 

Join our community of newsletter subscribers to stay on top of the news and at the top of your game. 

Paige Bailey, who began coding at age 9 in rural Texas, now leads the GenAI developer experience at Google. In a conversation with Chris Pirillo on The New Stack Makers, Bailey reflected on the evolving role of software development in the era of generative AI. While she once urged her nieces and nephews to pursue computer science degrees, Bailey now believes that critical thinking and problem-solving may be more crucial for future tech careers. 

She emphasized that generative AI is democratizing software development, making it more accessible and enabling developers to focus on creative tasks rather than the minutiae of coding. Bailey's experience at Google highlights this shift, as she now acts more as a reviewer and overseer of AI-generated code. She sees GenAI not as a replacement for developers but as a tool to accelerate their creativity and tackle longstanding backlogs. Bailey believes the key is ensuring everyone understands how to effectively apply generative AI to their work.

Learn more from The New Stack about the future of development: 

7 Ways to Future Proof Your Developer Job in the Age of AI 

The Future of Developer Careers 

4 Forecasts for the Future of Developer Relations

Join our community of newsletter subscribers to stay on top of the news and at the top of your game. 

Many organizations are finding that shifting to cloud native environments has become easier than it was in the past. However, the complexities and ensuing challenges can still surmount once at-scale deployments begin.

In this episode of The New Stack Makers podcast, hosted by TNS’ Alex Williams, founder and publisher, and Joab Jackson, TNS managing editor, application-deployment standards are the discussion of the day. The featured guests are Bruno Andrade, founder, Shipa, a provider of frameworks for Kubernetes; and Bassam Tabbara, founder and CEO, Upbound, which offers a universal control plane for multi-cluster management. 

The tech industry is broken. We deify overworking, and think burnout comes with bragging rights. But how do we break this exhausting cycle? In this episode of The New Stack Makers, we talk with LaunchDarkly's Manager of Developer Marketing Dawn Parzych about how to identify burnout in others and in yourself, how to treat it, and how to build a psychologically safe working environment that allows folks to say no.

With a masters in psychology and a DevRel role that certainly straddles people and tech, Parzych's work often sits on the people side of what they're building.

"I love the idea ofthe socio-technical systems that we're building,like tech, doesn't exist in a bubble. People are building the technology. They're very interrelated and you can't just focus on the tech, the people are the hardest part of tech. And we spend more time talking about how tech's the hard piece,where it's reallythe people and the interrelation betweenthe people and the machines," she said.

Wayfair describes itself as the “the destination for all things home: helping everyone, anywhere create their feeling of home.” It provides an online platform to acquire home furniture, outdoor decor and other furnishings. It also supports its suppliers so they can use the platform to sell their home goods, explained Natali Vlatko, global lead, open source program office (OSPO) and senior software engineering manager, for Wayfair as the featured guest in Detroit during KubeCon + CloudNativeCon North America 2022.

“It takes a lot of technical, technical work behind the scenes to kind of get that going,” Vlatko said. This is especially true as Wayfair scales its operations worldwide. The infrastructure must be highly distributed, relying on containerization, microservices, Kubernetes, and especially, open source to get the job done.

“We have technologists throughout the world, in North America and throughout Europe as well,”  Vlatko said. “And we want to make sure that we are utilizing cloud native and open source, not just as technologies that fuel our business, but also as the ways that are great for us to work in now.”

Open source has served as a “great avenue” for creating and offering technical services, and to accomplish that, Vlatko amassed the requite tallent, she said. Vlatko was able to amass a small team of engineers to focus on platform work, advocacy, community management and internally on compliance with licenses.

About five years ago when Vlatko joined Wayfair, the company had yet to go “full tilt into going all cloud native,”  Vlatko said. Wayfair had a hybrid mix of on-premise and cloud infrastructure. After decoupling from a monolith into a microservices architecture “that journey really began where we understood the really great benefits of microservices and got to a point where we thought, ‘okay, this hybrid model for us actually would benefit our microservices being fully in the cloud,” Vlatko said. In late 2020, Wayfair had made the decision to “get out of the data centers” and shift operations to the cloud, which was completed in October, Vlatko said. 

The company culture is such that engineers have room to experiment without major fear of failure by doing a lot of development work in a sandbox environment. “We've been able to create production environments that are close to our production environments so that experimentation in sandboxes can occur. Folks can learn as they go without actually fearing failure or fearing a mistake,”  Vlatko said. “So, I think experimentation is a really important aspect of our own learning and growth for cloud native. Also, coming to great events like KubeCon + CloudNativeCon and other events [has been helpful]. We're hearing from other companies who've done the same journey and process and are learning from the use cases.”

Is the end of programming nigh? That's the big question posed in this episode recorded earlier in 2023. It was very popular among listeners, and with the topic being as relevant as ever, we wanted to wrap up the year by highlighting this conversation again.

If you ask Matt Welsh, he'd say yes, the end of programming is upon us. As Richard McManus wrote on The New Stack, Welsh is a former professor of computer science at Harvard who spoke at a virtual meetup of the Chicago Association for Computing Machinery (ACM), explaining his thesis that ChatGPT and GitHub Copilot represent the beginning of the end of programming.

Welsh joined us on The New Stack Makers to discuss his perspectives about the end of programming and answer questions about the future of computer science, distributed computing, and more.

Welsh is now the founder of fixie.ai, a platform they are building to let companies develop applications on top of large language models to extend with different capabilities.

For 40 to 50 years, programming language design has had one goal. Make it easier to write programs, Welsh said in the interview.

Still, programming languages are complex, Welsh said. And no amount of work is going to make it simple. 

Learn more from The New Stack about AI and the future of software development:

Top 5 Large Language Models and How to Use Them Effectively

30 Non-Trivial Ways for Developers to Use GPT-4

Developer Tips in AI Prompt Engineering

DockerCon showcased a commitment to enhancing the developer experience, with a particular focus on addressing the challenge of debugging containers in Kubernetes. The newly launched Docker Debug offers a language-independent toolbox for debugging both local and remote containerized applications.

By abstracting Kubernetes concepts like pods and namespaces, Docker aims to simplify debugging processes and shift the focus from container layers to the application itself. Our guest, Docker Principal Engineer Ivan Pedrazas, emphasized the need to eliminate unnecessary complexities in debugging, especially in the context of Kubernetes, where developers grapple with unfamiliar concerns exposed by the API.

Another Docker project, Tape, simplifies deployment by consolidating Kubernetes artifacts into a single package, streamlining the process for developers. The ultimate goal is to facilitate debugging of slim containers with minimal dependencies, optimizing security and user experience in Kubernetes development.

While progress is being made, bridging the gap between developer practices and platform engineering expectations remains an ongoing challenge.

Learn more from The New Stack about Kubernetes and Docker:

Kubernetes Overview, News, and Trends

Docker Rolls out 3 Tools to Speed and Ease Development

Will GenAI Take Jobs? No, Says Docker CEO

In the rush to create, provision and manage Kubernetes, often left out is proper resource provisioning. According to StormForge, a company paying, for example, a million dollars a month on cloud computing resources is likely wasting $6 million a year of resources on the cloud on Kubernetes that are left unused. The reasons for this are manifold and can vary. They include how DevOps teams can tend to estimate too conservatively or aggressively or overspend on resource provisioning. In this podcast with StormForge’s Yasmin Rajabi, vice president of product management, and Patrick Bergstrom CTO, we look at how to properly provision Kubernetes resources and the associated challenges. The podcast was recorded live in Detroit during KubeCon + CloudNativeCon Europe 2022.

Rethinking Web Application Firewalls

Almost ironically, the most commonly used Kubernetes resources can even complicate the ability to optimize resources for applications.The processes typically involve Kubernetes resource requests and limits, and predicting how the resources might impact quality of service for pods. Developers deploying an application on Kubernetes often need to set CPU-request, memory-request and other resource limits. “They are usually like ‘I don't know — whatever was there before or whatever the default is,’” Rajabi said. “They are in the dark.”

Sometimes, developers might use their favorite observability tool and say “‘we look where the max is, and then take a guess,’” Rajabi said. “The challenge is, if you start from there when you start to scale that out — especially for organizations that are using horizontal scaling with Kubernetes — is that then you're taking that problem and you're just amplifying it everywhere,” Rajabi said. “And so, when you've hit that complexity at scale, taking a second to look back and ‘say, how do we fix this?’ you don't want to just arbitrarily go reduce resources, because you have to look at the trade off of how that impacts your reliability.”

The process then becomes very hit or miss. “That's where it becomes really complex, when there are so many settings across all those environments, all those namespaces,” Rajabi said. “It's almost a problem that can only be solved by machine learning, which makes it very interesting.”

But before organizations learn the hard way about not automating optimizing deployments and management of Kubernetes, many resources — and costs — are bared to waste. “It's one of those things that becomes a bigger and bigger challenge, the more you grow as an organization,” Bergstrom said. Many StormForge customers are deploying into thousands of namespaces and thousands of workloads. “You are suddenly trying to manage each workload individually to make sure it has the resources and the memory that it needs,” Bergstrom said. “It becomes a bigger and bigger challenge.”

The process should actually be pain free, when ML is properly implemented. With StormForge’s partnership with Datadog, it is possible to apply ML to collect historical data, Bergstrom explained. “Then, within just hours of us deploying our algorithm into your environment, we have machine learning that's used two to three weeks worth of data to train that can then automatically set the correct resources for your application. This is  because we know what the application is actually using,” Bergstrom said. “We can predict the patterns and we know what it needs in order to be successful.”

Moving workloads to the cloud presents cost prediction challenges. Traditional setups with on-premises hardware offer predictability, but cloud costs are usage-based and granular. In this podcast episode, Matt Stellpflug, a senior FinOps specialist at ProsperOps, discusses the complexities of forecasting cloud expenses with TNS host Heather Joslyn.

Cloud users face fluctuating costs due to continuous deployments and changing workloads. There are additional expenses for data access and transfer. Stellpflug emphasizes the importance of establishing reference workloads and benchmarks for accurate forecasting.

Engineers play a vital role in FinOps initiatives since they ensure application availability and system integrity. Stellpflug suggests collaborating with engineering teams to identify essential metrics. He co-authored an "Engineer's Guide to Cloud Cost Optimization," highlighting the distinction between resource and rate optimization. Best practices involve addressing high-impact, low-risk areas first, engaging subject matter experts for complex issues, and maintaining momentum. This episode also provides further insights into implementing FinOps for effective cloud cost management.

Learn more from The New Stack about FinOps and ProsperOps:

FinOps Overview, News, and Trends

ProsperOps Wants to Automate Your FinOps Strategy

Engineer’s Guide to Cloud Cost Optimization: Manual DIY Optimization

Engineer’s Guide to Cloud Cost Optimization: Engineering Resources in the Cloud

Engineer’s Guide to Cloud Cost Optimization: Prioritize Cloud Rate Optimization

Discover how large language models and generative AI are revolutionizing DevOps with PromptOps. The company, initially known as CtrlStack, introduces its unique process engine that comprehends human requests, reads knowledge bases, and generates code on the fly to accomplish tasks. Dev Nag, the CEO, explains how PromptOps saves users time and money by automating routine operations in  this podcast episode with The New Stack.

Dev Nag is joined by GK Brar, PromptOps' founding engineer, and our host Joab Jackson as they delve into the concept of generative AI and its potential benefits for DevOps. Traditionally, DevOps tasks often involve repetitive troubleshooting and reporting, making automation essential. PromptOps specializes in intent matching, understanding nuanced requests and providing the right solutions.

Notably, PromptOps employs generative AI offline to prepare for automating common actions and enhancing the user experience. Unlike others, PromptOps aims beyond simple enhancements. It aspires to transform the entire DevOps landscape by leveraging this groundbreaking technology.

Tune in to the podcast to gain deeper insights into this transformative approach that PromptOps brings to DevOps thanks to the power and possibilities of generative AI.

Learn more from The New Stack about DevOps and PromptOps:

DevOps News, Trends, Analysis and Resources

How to Use ChatGPT for IT Security Audit

What We Learned from Building a Chatbot

Kubernetes release 1.27 is boring, says Xander Grzywinski, a senior product manager at Microsoft.

It's a stable release, Grzywinski said on this episode of The New Stack Makers from KubeCon Europe in Amsterdam.

"It's reached a level of stability at this point," said Grzywinski. "The core feature set has become more fleshed out and fully realized.

The release has 60 total features, Grzywinski said. The features in 1.27 are solid refinements of features that have been around for a while. It's helping Kubernetes be as stable as it can be.

Examples?

It has a better developer experience, Grzywinski said. Storage primitives and APIs are more stable.