Artificial intelligence (AI) adoption ranks high on the list of IT investment priorities across industries. Yet, businesses remain cautious about going all in.
Generative AI is still a relatively new technology. Around 56% of executives view its integration in software development workflows as potentially risky. On the other hand, 46% of early AI coding assistant adopters claim a 5-10% revenue growth.
Industry leaders like Toyota, Coca-Cola, Mercedes-Benz, EY, and Capgemini have already implemented AI coding tools for in-house teams and reported significant productivity gains.
When it comes to AI in software development, it's all shades of gray. The end value will depend greatly on the selected use cases, team skillsets, and overall engineering process maturity.
In this post, we will discuss the benefits and limitations of AI in software development and share some best practices from early adopters.
AI in software development involves implementing new automation technologies, such as machine learning models, deep learning, and generative AI, to speed up everyday software engineering tasks.
In most cases, it assumes the adoption of AI coding assistants. These tools are based on large language models (LLMs) specifically trained to interpret, explain, generate, and edit code. Trained on a database of code snippets, these AI assistants can reproduce the syntax and coding patterns of the most popular programming languages. Some can also adapt the outputs to your coding standards.
These capabilities make AI coding assistants the talk of the town in the software engineering community. More than 70% of developers already use or plan to use gen AI tools as part of their development workflow, and almost 77% see this as a positive step. Below are the most popular use cases developers report.
Software developers are eager to adopt AI coding assistants because they can significantly accelerate and automate software delivery. These tools augment developers' skills and remove a lot of low-value, repetitive tasks.
For example, instead of generating boilerplate code or doing manual code reviews, developers can dedicate their time to writing new code, creating original solutions to problems, and learning new skills. Yet, there are many more use cases where software engineers use Gen AI tools, exploring innovative AI app ideas that can further streamline development processes.
GitLab provides the following data:
Source: GitLab
AI can potentially transform how fast and smoothly software engineering teams create, test, and deploy new software. According to McKinsey, developer productivity can increase by up to 45% when using AI coding tools. In particular, gen AI can accelerate code documentation by 45-50% and refactoring by 20-30%.
Early adopters report the following benefits of using Gen AI in their software development lifecycle:
Although there are gains, AI adoption in a software development process is not without stumbling blocks. And it’s far from being mainstream. Only 13% of organizations implemented AI in their software engineering workflows.
Why such a precaution?
of leaders, at least, worry about AI tools accessing sensitive customer information.
Source: GitLab
The same study says that 95% of C-level decision-makers prioritize AI coding assistants that can reliably protect their privacy and intellectual property.
Some other valid concerns of AI-assisted coding include:
Like any other tech innovation, AI adoption for software engineering doesn't automatically lead to increased productivity, speed, profit, or other positive outcomes. The effectiveness of an AI coding assistant depends on how you implement and use it, bringing out its strengths and managing its limitations.
The adoption of AI in software development requires careful planning. It’s not enough to select the most popular AI coding assistant, demonstrate it to your development teams, and then expect results in several months. You must train your engineers, implement robust security controls, and establish governance standards. This is no easy task, especially for larger IT departments.
The key to ensuring that AI benefits your entire software delivery process is to adopt it strategically, starting with a small-scale pilot project and gradually expanding it to the whole company.
Start with a proof of concept deployment like Duolingo and PayPal did, then work up to scaling your AI-assisted engineering program. Here’s how to do so:
Identifying your reasons for adopting AI is the first step. Be specific and think about desired outcomes. Do you want to launch more features, reduce tech debt, or curb the volume of security incidents? A clear, measurable objective will help you quantify the results.
Make sure your AI implementation strategy adds value both to your software engineering team and executive leaders. There must be a willingness to accept change from developers and a green light for initiative from C-level representatives. Here’s a tip for each group:
AI-assisted development tools aren’t in short supply. There are enterprise offers from big tech companies like Amazon Q Developer or Microsoft Copilot, as well as innovative products from seed-funded startups like Wispr and Replit. Some products offer specialized security coding and testing tools, while others cover general software development tasks.
Here are the top 10 code assistant tools used by professional developers as of May 2024:
Source: Stack Overflow
With so many options available, selecting an AI code generation tool can be tricky. To simplify the process, keep these criteria in mind:
Many AI coding tool vendors offer demos, free trials, or freemium versions of their products, so you can test several options before committing.
AI coding assistants need the right input from your developers — a solid code base and relevant conversational prompts. Developers will also need to determine if the suggested code snippets actually work, have no security vulnerabilities, and don’t change system behavior or business logic.
That’s why, for your pilot projects, it’s best to select senior or middle-level software engineers with sufficient skill to evaluate the quality of AI outputs.
Also, provide adequate AI training for your pilot team. AI coding assistants are a relatively new phenomenon, and even experienced developers might not be familiar with their functionality. Also, you might need to train developers on best practices for prompt writing (accurate task descriptions, project context, expected output, etc.).
AI coding software vendors offer some solid onboarding support. GitHub Copilot has video tutorials and a community portal. Tabnine shares how-to videos on YouTube, and Codeium created a Chrome extension to test the tool directly in the browser.
Before launching the pilot program, you must decide on the metrics for measuring whether the AI assistant benefits or hinders the productivity of your development team.
Calculating the sheer volume of performed work might be misleading: a lot of bad-quality, unreadable code is not conducive to good pilot project performance. Measuring code quality on its own also has limitations. To ensure AI-generated code works well, your developers might have to spend a lot of time testing and editing it.
Instead, select a more insightful approach to measure performance from different angles, as PayPal did during their early adoption of DUET (now Gemini). They evaluated:
Unsurprisingly, PayPal's metrics match those that developers themselves suggest using for performance measurements. Code quality and time are the top two criteria, according to the tech community surveyed by GitHub.
Source: GitHub
Introduce AI tools to a limited group of developers and focus on specific use cases (code generation, refactoring, boilerplate coding, etc.). Later, you can apply acquired knowledge and experience to either scale the initiative company-wide, select another platform, or even postpone the adoption until the technology matures.
Form two test groups — an AI-assisted cohort and a regular one. Assign each team with similar tasks and make sure they have the same starting point (like clean GitHub and Jira Instances).
To ensure your pilot project is secure, PayPal’s senior director of product management, Jarred Keneally, suggests working in isolated sandbox environments and practicing on non-proprietary code. Also, make sure to continually measure performance against the previously established criteria.
Accenture started testing GitHub Copilot with just 20 developers. Then, the company decided to hold a randomized controlled trial (RCT), measuring the performance of 450 developers who used AI and comparing the results to 200 developers who did not. Accenture saw an increase in successful builds, pull requests, and software updates, further unrolling the AI initiative for 12,000 developers.
Continuous feedback loops are crucial to understanding if the selected AI coding tool works for your use case. Collect both positive and negative feedback from pilot project participants — and do so throughout and after the project.
For a 360-degree view of the pilot, collect both qualitative (interpretation-based experience of your teams) and quantitative data (measured time, quality, and volume of code). We recommend the following activities for measuring results:
Most importantly, be on the lookout for productivity loopholes, where the AI tool improves performance in one aspect but adds significantly more work elsewhere. For instance, code generation time may be down, but developers may now have too much testing to do.
The GitHub resource portal also suggests the following metrics to measure the impact of an AI coding assistant: suggestions delivered, number of acceptances, lines of code accepted, total completed pull requests, time to merge, total successful builds, and others.
EY, for instance, measured the success of their AI adoption pilot projects in lines of code committed. Within several weeks, 150 developers generated more than 1.2 million lines of code with the help of an AI coding assistant.
Meanwhile, the National Australia Bank measured code acceptance rates, indicating that their developers took in 50% of code suggestions generated by an AI tool.
Gradually scale the initiative department-wide, training more developers, covering new use cases, and creating onboarding materials for them. Continue measuring the performance of your development teams and gather feedback to see if the program is scaling well and bringing expected results long-term.
In time, you might want to scale your AI adoption for the whole software development lifecycle. As per KPMG, AI-powered tools can be used at every stage, from planning to maintenance. Here’s how:
AI adoption also requires a solid legal framework. Make sure the tools, methods, and practices you adopt do not violate AI regulations. EU's AI Act, for instance, applies to all EU-based natural or legal persons using AI systems professionally.
To ensure the safe and ethical use of AI in your software development workflows, consider assembling a dedicated AI committee. This working group, consisting of experts in law, compliance, privacy, and information security, will guide you in developing, testing, and deploying AI-generated code responsibly.
Initially, this group can help you analyze the security and privacy settings of selected AI coding assistants. It can suggest whether you need to have extra legal clauses with your clients or vendors — some may restrict the use of certain data or tools, while others might forbid using AI at all.
Finally, the committee can help you create a code of conduct for the human-centered use of AI.
Generally, a human-centered approach to using AI in software development assumes:
Generative AI can add value to your software development processes, reducing routine tasks and empowering your teams to work smarter, not harder. What’s more, the technology offers almost limitless flexibility and scaling potential.
McKinsey states that market leaders use gen AI in three or more business functions while their less prominent competitors use it in two. Once you’ve seen the potential of AI based on the software development pilot project, you can consider bringing automation to other functions like marketing, sales, and HR.
To streamline work further, collaborate with a strategic technology partner who can provide consulting and specialized knowledge on AI adoption — not only for software engineering but for all organizational functions. Select someone you can trust to guide you in the right direction and help you create and execute the whole AI adoption plan.
Find out how AI is setting new standards and driving the future of software testing!
Edited by Jigmee Bhutia
Gone are the days when writing test automation meant hours of coding and debugging.
by Geosley Andrades
Low-code and no-code technology enable businesses to meet the ever-increasing software demand,...
by Mika Vice
The rise of no-code development ushered in a new era in software development.
by Abhinav Girdhar
