GitHub, the online developer platform that allows users to create, store, manage, and share their code, has been on a generative AI (genA) journey since before ChatGPT or Copilot was widely available to the public.
Through an early partnership with Microsoft, the dev platform adopted Copilot two-and-a-half years ago, tweaking it to create its own version — GitHub Copilot.
The genAI-baed conversational chat interface is now used as a tool for both GitHub users and internal employees to assist in code development, as well as an automated help desk tool.
There are people who believe as genAI continues to evolve and can produce more code based solely on user requests, developers will no longer be needed. As Nvidia CEO Jensen Huang last week said, because of AI, “everybody in the world is now a programmer. This is the miracle of artificial intelligence.”
Instead of software development, Huang believes humans should focus on more important skills such as biology, education, manufacturing, or farming, and the language of programming is now the human language.
GitHub
GitHub's COO Kyle Daigle
Kyle Daigle, who's worked at GitHub for 11 years, took over as its COO about a year ago. He’s been part of a genAI development strategy that’s focused on discovering how the technology can benefit its roughly 3,000 employees — developers and non-developers alike — and its external developer community of users. So far, genAI is making developers 55% more productive, he said.
Daigle spoke to Computerworld about the various ways genAI has created efficiencies and helps both developers and non-developers. The following are excerpts from that interview:
When did you deploy Copilot. Why? And what has it enabled GitHub to do? "We’ve been on a journey with Copilot for about two and a half years now. We started working on Copilot when we got early access to the OpenAI models through our partnership with Microsoft. Similar to a lot of companies now, the main question was how do we put these LLMs to good use? It took us a little while to figure out the secret sauce that’s now Copilot. Originally, when we were using the models, we thought we were going to be building tools that documented code. You know, you give it your repository and it would spit out what that code did.
"But through experimentation, the idea of Ghost text — the sort of completion model of what Copilot does, where it shows you the entirety of a single message versus a single line — was kind of a big breakthrough for being able to get the most out of a powerful tool. And so, fast forward and now we’ve got more than a million GitHub users using Copilot every day. Our stats show it’s making them 55% more productive, and it’s writing about 60% of code; we expect that to get up to about 80% of code over time in many languages.
We do find it actually makes people more productive, happier and ultimately removes their toil.
"I think most importantly, and something we talk to our internal teams a lot about, is that it’s making developers feel more fulfilled. It’s allowing them to do more creative work and not the toil work. Instead of looking for the genAI to do the creative work, we’re allowing the human be in the pilot’s seat as the developer.
"So, we’ve had a lot of success during this time in getting every dev in our world allowing GitHub in helping them to write code.
"Internally, we’ve been focused on taking the lessons from Copilot and applying that to other places where we use AI tools, including Copilot outside of software development use cases. And of course other purpose-built tools as they come to market to help us all be a bit more productive."
Assisting in code development seems to be one of the earliest low-hanging fruits your genAI platform addressed. How long have you been using it to assist in code production, and what languages was it helping with? "In our early experimentation, we were doing a lot of work in Python, JavaScript and languages like that. GitHub is mainly a Ruby company, but we also write in Go, and C, and FirGit. And so we were expanding our use cases of Copilot and using it in different languages. But overall, Copilot is able to work on the vast majority of languages that are in the public sphere.
"If you have a proprietary language, it can emulate that language because it’s looking at the code inside your repository and it can do a pretty good job of figuring out what it needs to use to feed the next line of code or next method.
"So, we’ve gone from a couple of test languages to essentially every modern programming language that has enough context in open source and the Internet."
How is Copilot doing in terms of completing code? "In terms of code completion rates, what we’re talking about is in some cases when writing with Copilot, it might finish one line of code, but it might also finish an entire method; it might finish an entire file or class depending on the language you’re in.
"With something like Copilot Chat, you can talk to Copilot and say, 'This is the problem I’m trying to solve,' and it can potentially generate the entirety of a file for you. You can then say, 'I don’t want it to be blue, I want it to be red. Or I want it to use this API or that API' — those sort of adjustments.
"When we talk about the amount of code generated, we’re talking about the amount of code Copilot gave and the amount the user kept over time. Obviously, when you get a completion, you may think, 'Oh, that’s not right.' Or if you’re a developer you may have it write out a bunch of code and then realize I can refactor it right now; this isn’t quite right.
Sometimes the best code is no code at all and it’s just using human language.
"What we’re seeing is the vast majority of the code generated by Copilot is kept. Then down the line, after you write codes, you submit a PR, you run continuous integration, the next steps are also faster. So, code review on code that was written by Copilot with a developer tends to be faster because the code ends up being more correct. Continuous integration ends up being green more often than red on the first build because the code tends to be more correct. So there are a lot of interesting impacts downstream, too, when you’re able to use Copilot as part of your workflow as a dev."
Have you seen any issues — inaccuracies — while using Copilot? "The code that we’re providing you is also mimicking the code it sees in your repository. So in some instances, if it has an older code base, it’s going to take that into account and maybe do a practice that’s no longer modern, like the casing of variables or if you have a library that it’s going to call because it’s trying to emulate your existing project along with the underlying model. So for some developers, how they do that is use Copilot chat to say, 'Hey, we’re actively upgrading this right now, so I’d like to use the new approach.' And that’s one way we get through it.
"But also, we’re taking a lot of the technology we have with Copilot and the AI models we have underneath it and also finding other places we can help developers remain secure by fixing vulnerabilities as well. So, one of the things we shared at the GitHub Universe conference in November was something called Security Autofix, which uses similar underlying AI technology. So when tell it you are aware of a vulnerability...in a code base, we’re not just going to tell you you have a vulnerability you need to fix; we’re also going to do the fix right there. All you have to do is say, 'Yes, I’m good to go.'
"So, Copilot is still in the ‘copilot seat.’ You still need to follow best practices. You still need to do security scanning and secret scanning and all the things that have been true for good software development practices, regardless. But we’re trying to make ways to bring AI throughout the entirety of the software development lifecycle on GitHub to help you out. So it’s not just in your IDE when you’re writing code."
Some in the tech industry are concerned that AI’s ability to automatically generate code could eliminate developers. What’s you take? "I think throughout modern history, there are so many moments when a piece of technology has come into the world, like the printing press, and everyone was like, hold on a minute —where’s my job going to go? But really what happened is a lot of work and opportunities that weren’t economical anymore were suddenly economical, because you’re not having your developers spend 60% ro 70% of their time solving problems that have been solved dozens, hundreds and thousands of times.
What we’re seeing is the vast majority of the code generated by Copilot is kept.
"So, the reality is...GitHub Copilot is making developers 55% more productive. Some customers have the reaction, 'Does that mean I just get 55% of my developer’s time back?' The reality is what your developers get back is the ability to solve harder problems more creatively than they previously had been able to when they had to do all that rote work themselves.
"So at GitHub we’re still hiring developers. We’re hiring right now. Really, what we find is we’re focusing more time on that upfront discussion or architecture and the problem we’re solving with customers. Because as the coding becomes faster, it becomes more important that you're spending your time doing that creative problem solving work rather than that rote work that we’ve all done at some point in our career.
One of the major pieces of Copilot that goes under appreciated is the ability to do learning and development, or upskilling on the job.
"I’m more excited about the opportunity AI is starting to give us, which is we can do more than we did before because we’re able to write more code than we have previously. So, we can solve bigger problems that maybe before wasn’t possible, like that big re-write of your app. So many customers put that off and put it off because how would you ever be able to afford it. But, if it’s 50% less expensive, well maybe you can move to that new technology and you can use that new technology to move even faster in solving the next set of problems.
"I think we have plenty of work left to do. By every measure of every analyst in the world, there’s not enough developers in the world. So, we still have quite a gap to go before we have to worry about developers not having enough work to do."
What benefits are non-developer teams seeing from genAI? "One of the major pieces of Copilot that goes under appreciated is the ability to do learning and development, or upskilling on the job. You have someone that’s new to a role, a company, a language and being able to come in and immediately have someone there you can ask questions of, that you can write code with and get that immediate feedback loop with Copilot.
"It’s not just the new folks. It’s also in a lot of cases those most senior developers we have where they get put in those nasty old projects that are extremely important to the business but have been just sitting on the shelves in the back closet somewhere keeping everything running. When they need to go into those projects and make updates, that L&D side has been really huge because they can go in and say, 'I know Java, but I don’t know Scala. Or I know Java, but I don’t know .Net,' or whatever. And, can you help me know what the next steps are?
"In a similar vein, we also focus Copilot on that user experience, i.e., you just start typing. There’s no real enablement; there’s no learning, there’s no buttons to figure out.
"So, I took those two learnings and we started to look internally into other places where we thought there was toil and where we could implement AI without having to enable AI. That’s the real secret. If you have to teach people how to use it, it’s not much better than any other technology choice you could make.
So, what were your other big wins? "One of the first big wins we had at GitHub, and I’ve seen this at other places, is in taking AI and bringing it into the IT setting. We’ve got a little over 3,000 Hubbers [GitHub employees] that have entered in hundreds and hundreds and hundreds of [helpdesk] tickets in an old system, to get assistance on why their laptop isn’t working, how they get access to the VPN, and so on.
"GitHub essentially runs in Slack. We’re a remote-first company. We’ve got employees all over the world; we’re not a return-to-office company. What we did was say, if we’re all on Slack, why not make that interaction with IT in Slack also be AI powered?
"So, instead of going into a portal and submitting a ticket, we have a channel called IT Help Desk, and in that channel is a bot that we call OctoBot. And when you ask a question, a vendor called MoveWorks that we partnered with ... sees that question and OctoBot will come and say, 'Yeah, I know exactly what you need to do. Here are your next steps.' And in many cases we can even automate the workflow to say..., 'we’ll go set this up in these other systems for you.'
"Maybe it’s what every developer wants and it’s your laptop upgrade day and you’ve waited your two years. Click this bot and we’ll ship you your new laptop, and because we didn’t create a new system and didn’t have to teach anyone about a new portal, we’ve seen a huge improvement. We’ve gotten to where OctoBot is solving 30% of these tickets right out of the gate. We’re saving hours per day of every IT staff member's time that we’re able to reinvest in other AI initiatives.