AI-Assisted Development: From Augmentation to Transformation by Adani Arisy from Nalarin.com

This presentation, "AI-Assisted Development: From Augmentation to Transformation," by Adani Arisy, an independent technology consultant from nalarin.com, explores the growing impact of AI on the software development industry.

The slides argue that using AI effectively is becoming a fundamental expectation across the industry, with companies like Shopify and Zapier integrating AI into their core operations and performance reviews. The presentation explains that modern Generative AI, or Large Language Models (LLMs), function as "Super Autocomplete" that predicts the next token rather than truly understanding content. It highlights the importance and limitations of the "context window," which is likened to a finite whiteboard space, and discusses challenges like "Lost in the Middle" and "Context Rot".

Key application areas for AI in coding are outlined, categorized by their degree of autonomy:

Code Review (Low autonomy), using tools like CodeRabbit.

Pair Programming (Medium autonomy), with tools such as Cursor and GitHub Copilot.

Delegation Agent (High autonomy), featuring examples like Claude Code and OpenAI Codex.

For successful organizational adoption, the presentation recommends several best practices: establishing clear policies and onboarding for AI tools, creating shared resources like prompt libraries, and implementing monitoring and measurement systems. It also provides a detailed example of custom instructions for a GitHub Copilot agent to improve prompt quality.

The discussion extends beyond coding, showing how AI can provide fast feedback and reduce collaborative overhead in various phases of development. Examples include using AI for:

PRD and RFC Reviews to critique value propositions and analyze architectural trade-offs.

Test Case Generation to ensure exhaustive coverage and prevent bugs.

Backlog Grooming to generate detailed task descriptions from PRDs and RFCs.

Finally, the presentation looks ahead, suggesting that roles will likely compress (similar to how DevOps emerged) and that the industry is shifting from a "writing-first" to a "building-first" culture where AI enables rapid prototyping. It concludes with the idea that while AI won't replace humans, "humans with AI are going to replace humans without AI".

1. AI-Assisted
Development FROM AUGMENTATION TO TRANSFORMATION Adani Arisy NALARIN.COM AI-Assisted Development Page 1 of 16

2. Hi, I’m Adani [email protected] 9 I’m an independent technology consultant and
solopreneur building consumer-facing mobile apps.
I have over 11 years of software engineering
experience, having recently served as VP of
Engineering at Mekari and previously at Grab. Adani Arisy AI-Assisted Development Page 2 of 16

3. AI is Eating the Industry NALARIN.COM 5 Using AI effectively is now a
fundamental expectation of
everyone
AI must be part of prototype phase
Before requesting headcount,
teams must prove AI can't do the
job CEO Shopify - @tobi X.com AI-Assisted Development
Shopify CEO memo is only one from many other companies trying to leverage the power
of AI to level up their performance.
Incorporating AI into performance review and many decision making process.
Page 3 of 16

4. New Reality NALARIN.COM 5 Rising expectations all across
the industry
Company establishing policy to
enable AI upskilling and enable
AI adoptions From AI-friendly to AI-first: How Zapier is transforming hiring and onboarding AI-Assisted Development
In Zapier, new hires are expected to have certain degree of AI fluency.
It is seen as signal of a high value employee.
Page 4 of 16

5. LLM = Super Autocomplete It predicts the next
token (piece of text)
over and over NALARIN.COM 4 Feels smart because it’s
trained on huge corpora Not a brain. Not a
database. It’s a predictor PREDICTION ≠ FEELINGS OR UNDERSTANDING AI-Assisted Development
Modern AI is mostly referring to Generative AI using transformer architecture.
It sounds super smart and mimic human really well compared to previous generation of AI.
However to maximize our utilization of AI, it is important to know what exactly it is and
isn't.
Think AI like a parrot which can be trained to mimic human speech but it does not actually
understand what it exactly means.
Like when it says "Assalamualaikum"
Page 5 of 16

6. Context window = your whiteboard space NALARIN.COM 7 Remember: You get a finite space. More text ≠ better good placement > dumping Lost in The Middle models recall the start/end best Context Rot / Pollution old, messy threads reduce signal ✓ Ask the AI what context it needs to give you the best answer.
✓ Be mindful of AI limitations when analyzing large codebases or documents.
✓ If a long conversation goes nowhere, restart the chat for a fresh perspective. AI-Assisted Development
The most important attribute and most the limitation of the model is revolving around the
term Context Window.
Think of it like a whiteboard
A lot of people says when you provide it with more context it will perform better. It is
generally true but is not always the case. Give example:
Lost in the middle problem is based on recent paper from researcher stanford & university
of california.
Context rot is what happens when context gets so big and the output quality starts
degrading.
<Show below box>
The whole context limitation and LLM behaviour can be quite complex. We used to hear
the name "prompt engineering" used to describe a new role. But a more accurate name is
Context engineering. How to properly inject context to LLM to accomplish the goal.
Page 6 of 16

7. Mode Example Tool Degree of Autonomy Code Review CodeRabbit Low Pair Programming Cursor, Github Copilot Medium Delegation Agent Claude Code, OpenAI Codex High Using AI in Coding NALARIN.COM 8 The hottest enabler of
AI adoption.
3 most common usage
patterns. AI-Assisted Development
The most common usage of generative AI in tech company is obviously to generate code.
Nearly every software engineers must have used some of these tools.
Page 7 of 16

8. Organization level policy and onboarding for AI
tools 1 Shared practices (e.g. prompt library, project
level custom rules / agent instructions) 2 Monitoring & Measurement 3 Best Practices NALARIN.COM 3 AI-Assisted Development
When a company decides to equip their engineers to use AI tools, there are some best
practices to ensure that a company can maximize the value they get from this tool. This is
important because these tools has a relatively high cost.
1. Without org level policy, people will be confused on what tool to use, how to get access
to it, and sometimes even how to use it.
2. Once it is used, we might want to standardize some shared practices, at least on project
level
3. Organization that monitors and measure the usage will be able to identify if there's any
problem about the tool/adoption. Those who don't might miss out on the opportunity to
improve and stuck on getting low value.
Page 8 of 16

9. Example Custom
Instructions NALARIN.COM 5 AI-Assisted Development
What custom instructions should be used will depend on the team's practice and the kind
of problems most individuals are facing when using the tool.
For example, this is a sample custom instructions for a team that has yet to be fluent to
prompting agent mode and usually only writes a couple sentences to instruct the AI.
This is a sample custom instructions that can be used to encourage the team to step up
their prompting/context engineering game.
Page 9 of 16

10. AI Beyond
Coding NALARIN.COM 6 Reflexive use of AI is one of the lowest hanging
fruit in enabling fast feedback and reducing
collaborative overhead AI-Assisted Development
Using AI coding assistant is all neat and good. But to truly unlock the power of AI, ti does
not stop there.
If you are a solopreneur like me, using AI coding tools will easily 10x your speed. This is
because I do everything myself when developing a product. There's no one I should ask
when deciding what to build and how to build it.
But it is not the case when you are a part of cross functional software engineering team.
There will be a lot of back and forth to align understanding, get feedbacks, etc.
Those processes are important but can be time consuming. AI will not automatically get
rid of it because alignment in understanding is still important, but it can help reduce
feedback cycle.
Page 10 of 16

11. Phase How AI Can Help Benefit PRD Review Critic value proposition, review
requirement completeness, and
point out project risks Reduce Back and Forth with
Manager / Stakeholders RFC Review Review architectural trade off, point
out security / performance risks /
missing error handling Reduce Back and Forth with
Manager / Stakeholders Test Case
Generation /
Review Generate well structured test
cases, review completeness /
missing edge cases Ensures test cases are
exhaustive. Prevent production
bug. Backlog
Grooming Generated detailed task
descriptions / breakdown based on
PRD & RFC Shorten backlog grooming
meeting. Reduce incomplete
information for implementation. Fast Feedback With AI NALARIN.COM 8 Utilize AI as thought
partner and reduce the
amount of back and
forth. AI-Assisted Development Page 11 of 16

12. Example: AI in RFC Review NALARIN.COM 10 Draft RFC Feed RFC document
(.md or PDF) to AI Pre-Human
Review Revision PIC Address AI
feedbacks. Describe
the changes taken on
the RFC changelog During Review PIC would describe
to reviewers how AI
helped with the docs Giving Approval Ensure AI and Human
feedbacks are
properly addressed AI-Assisted Development
- Recommended to feed in form of PDF to ensure that diagrams can be included as part of
inputs
- Self enforcement in form of including it in the document and presenting it during review
meeting. Also has the benefits of sharing learnings.
Page 12 of 16

13. Test Case Creation / Backlog Grooming NALARIN.COM 8 Use AI formula in
Google Sheet or
Excel to easily
mass generate task
description & test
cases AI-Assisted Development Page 13 of 16

14. Monitoring & Measurement NALARIN.COM 7 Ensure AI practices delivers value to overall product development pipeline Tools Usage Metrics Example:
Seat Occupancy
Suggestion Acceptance Rate
Daily Active User
Tools like Github Copilot / Cursor has
built in reporting dashboard / API to
obtain these data
Use to diagnose tool adoption within
the team. e.g. Low suggestions
acceptance rate could mean the team
is not able to effectively prompt or
maybe the tool is bad. Self-Reported Metrics Example:
Number of PR Created mostly by AI
(self-labelled)
AI Trust Level (survey)
AI productivity impact (survey)
Example Survey:
https://dora.dev/research/ai/gen-ai-
report-questions/
Be careful not to set target /
performance incentive based on self-
reported metrics (i.e. perverse incentive) Frequent Check In &
Retrospective Create AI Champions program to
promote AI usage internally and
empower people who are enthusiastic
to create impact within the org
Incorporate AI topics into team
retrospective to share interesting use
cases and failures.
Check-in individually with the team to
get their overall sentiment on AI
initiatives and individual AI fluency. AI-Assisted Development
- Measuring don't have to be super complicated with telemetry and advance monitoring.
- Self reported metrics can be as effective as long as we can prevent perverse incentive
- Collect signals via retrospective, internal interview, or surveys
Page 14 of 16

15. AI is not going to replace humans,‌
but humans with AI are going t‌o
replace humans without AI‌ KARIM LAKHANI, PROFESSOR AT
HARVARD BUSINESS SCHOOL Look Ahead 5 We’re still relatively early and best
practices changes frequently
However, it will definitely impact out
way of working profoundly @REALMADHUGURU - HEAD OF PRODUCT @ GOOGLE AI-Assisted Development
Role compression will likely happen.
Just like how cloud technology and containerization is compressing the roles of infra
engineer and release engineer into a single DevOps role.
Page 15 of 16

16. The End THANK YOU Adani Arisy [email protected] AI-Assisted Development Page 16 of 16