Migrating legacy systems is rarely just a technical problem. It is also a problem of ownership, perception, team habits, deployment confidence, and operational boundaries.

In one of our projects, we had a few legacy services written in Python. Over time, we decided to migrate to Go — not just to change the language, but for better maintainability, stronger typing, improved performance, and a stack that aligned with our long-term engineering direction.

At first, our migration strategy looked simple: build the new service in Go, keep Python running, and do all the new development in the new service. Existing functionality would be migrated gradually as tech debt. While on paper this sounded safe, in practice, it created several problems.

The Initial Approach: Parallel Services

The initial plan was to run Python and Go side by side. Python would serve existing functionality. Go would handle new features. Over time we would migrate older functionality on demand or as a tech-debt task.

The split was made explicit through versioned endpoints. /v1/ routes pointed to the Python legacy service. /v2/ routes pointed to the new Go service. Clients, whether internal services or external-facing load balancers were written against whichever version was convenient at the time, with no strong rule about which should be preferred.

The architecture looked like this:

This felt low-risk. We were not touching much of the old system at once, and the new service could be built properly without blocking existing business flows.

But the versioned split introduced a subtler problem than we anticipated. Because both versions were equally valid entry points, clients had no incentive to migrate to /v2/. New clients picked whichever version had the endpoint they needed. Existing clients stayed on /v1/ indefinitely. The version number became a permanent label rather than a migration signal.

The Problems with Parallel Services

The old service stayed primary — by default

Even with Go running, engineers naturally reached for Python whenever a feature touched existing business logic. It already had the database access, queue integrations, and production-proven behavior. The new service existed, but it was never the default.

As long as Python remained the easiest place to make changes, migration naturally slowed down. Tech debt rarely gets prioritized without ownership or urgency.

A migration does not move forward just because a new service exists. It moves forward when the new service becomes the default path for development.

Shared functionality blurred boundaries

Features rarely align neatly with service boundaries. When a new Go feature needed logic that already existed in Python, the team faced a recurring decision:

• Duplicate the logic in Go?
• Call the Python service from Go?
• Migrate the shared logic first?
• Just modify Python because it is faster?

Most of the time, modifying Python was fastest. Every such decision quietly made the migration harder and the service boundaries blurrier.

Datastore ownership was never defined

Which service owned the database schema? If Go needed a new table related to old behavior, where did the migration live? If a queue was produced by one service and consumed by the other, who owned the event contract?

Without clear ownership, every datastore change became a discussion. The architecture did not define ownership, so the team had to negotiate it each time.

The Shift: Making the New Service Primary

The real change happened when we flipped the model.

Instead of treating Go as a parallel service, we made it the primary service and Python as an auxiliary service. Requests came to Go first. If functionality was already implemented there, Go handled it. If not, it fell back to Python.

All interactions with the Python service went through the Go service. Clients no longer needed to know whether a capability lived in Python or Go. Go became the single entry point, while the decision to handle a request locally or fall back to Python remained internal to the service.

The old service still existed, but it became a fallback — not a co-equal sibling. Feature work stopped defaulting to Python. The migration became a real project with real momentum.

The question shifted from:

“Should we build this in the new service?”

to

“Is there any reason this cannot be built in the new service?”

That small change in framing fundamentally changed how engineers approached the system.

The Approach: Incremental by Design

The new service acted as the primary router. It decided whether a request should be handled by the new implementation or forwarded to Python. This gave us flexibility. We could migrate one endpoint, one use case, or one customer flow at a time. No big-bang rewrite.

This gave us two useful controls. The feature flag answered the binary question, has this route been migrated yet? The rollout percentage answered the confidence question — how much real traffic is the new implementation ready to handle?

The same routing logic could have been pushed to the client level — each caller decides which version to hit. But that scatters the migration across every clients, makes the rollout percentage harder to enforce consistently, and means any change to the strategy requires coordinating updates across multiple codebases. Keeping routing centralised in the Go service gave us a single place to change, monitor, and reason about — one config change affected all traffic, not just the callers that had been updated.

In practice, a route would start at 5% rollout, run for a few days under observation, then ramp to 25%, 50%, and eventually 100% as error rates and latency held steady.

The rollout assignment also needs to be deterministic. Instead of randomly switching between implementations for every request, traffic can be assigned using a stable identifier such as a customer, account, or tenant so the same entity consistently follows the same path during the rollout.

If something looked wrong at any stage, the percentage could be dialed back without a deployment. Rollback was a config change, not an incident.

This was our operational version of the Strangler Fig pattern: progressively transferring ownership from Python to Go while keeping the legacy implementation available as a controlled fallback.

Success Criteria: Observability Over Assumptions

As we started migrating traffic, we needed to understand how the new implementation was performing so that decisions could be driven by data rather than assumptions. We instrumented every request at the routing layer, tagging each one with the route, the decision (handled in Go vs. forwarded to Python), and how it performed. Those metrics were ingested into Prometheus and monitored periodically through dashboards to check the migration status.

The metrics we monitored most closely were:

• Traffic split per route: What percentage of requests actually landed in Go vs. Python, plotted against the configured rollout percentage.
• Error rate, Go vs. Python, same route: A spike in Go’s error rate (4xx, 5xx) immediately after a rollout bump was the signal to dial the percentage back down.
• p50/p95/p99 latency, Go vs. Python: Migrations are also a performance bet. We wanted proof Go was at least as fast before pushing past 50%.
• Stale flags: A simple query for flags sitting below 100% with no rollout change in the last N days. This caught flows that had quietly stalled at 25% for weeks with nobody driving them forward.
• Routes with no flag at all: Anything still defaulting to Python with zero migration activity. This was the literal scoreboard: “43 of 71 routes are live in Go” came directly from this query.

None of this needed to be fancy. A handful of dashboard panels and a couple of scheduled queries against the feature flag configuration were enough to turn “we think the migration is going well” into a number anyone on the team could check.

A route was not considered migrated when it reached 100% Go traffic. It was considered migrated only when the fallback path, feature flag, and corresponding Python implementation were removed.

That distinction was important because 100% rollout only told us where traffic was going. Removing the fallback told us that ownership had actually moved.

Database and Queue Ownership

With parallel services, this had been deliberately left vague. Migrations were a coin toss depending on who was deploying. When we made Go the primary service, we drew a hard line: Go owns all database migrations and new queue or topic creation.

Migrations ran through a pre-upgrade hook in the Go service’s Helm deployment. Python could continue reading and writing application data where required, but it no longer owned schema evolution. All migrations originated from the Go service.

This forced a useful discipline. Any schema change went through Go’s migration files first, was reviewed alongside Go code, and deployed with the Go service. No ambiguity about order of operations.

Rolling back the Go service was safer than a typical dual-write scenario because Python never owned or applied migrations. However, schema changes still needed to be backward-compatible. Additive changes like new nullable columns remained compatible with an application rollback; destructive changes like column renames or drops were not.

The same principle applied to queues and event contracts. Infrastructure ownership followed the target architecture: new topics, queues, and event contracts belonged to Go, while Python integrations were treated as temporary producers or consumers where legacy functionality still depended on them.

For each integration, we wanted the ownership to be explicit: which service owned the event contract, which producer was authoritative, whether both implementations could safely consume during migration, and what condition allowed the Python producer or consumer to be removed.

That prevented temporary messaging paths from quietly becoming permanent parts of the architecture.

The Risk: Fallback Can Become Permanent

Fallback is useful during migration, but it has a risk — it can become too comfortable.

If the new service simply forwards too many requests to the old service, the migration may look complete from the outside while the old system still does most of the work. We have not really migrated the system. We have only hidden the old system behind a new entry point.

Every fallback path should have an owner, a metric, and a removal plan.

Otherwise, the fallback layer becomes the new legacy layer.

Conclusion

Our initial plan of running Python and Go in parallel looked safe, but it did not create enough migration momentum. The old service kept receiving changes. Shared functionality caused confusion. Infrastructure ownership stayed unclear. Migration became optional tech debt instead of a natural part of development.

The migration became more effective when we made Go the primary service and used Python as a fallback. That shift changed both the architecture and the team’s mindset.

A new system does not replace an old system just because it exists. It replaces the old system when it becomes the default path for change.

Note: Some project-specific details, such as AWS regions and internal identifiers, have been intentionally abstracted for anonymity. The architecture, trade-offs, and implementation approach remain close to what we built.

Disaster Recovery is one of those things every engineering team debates. For most platforms, multi-AZ is a reasonable answer — it handles datacenter-level failures and keeps the system running. Going further wasn’t a choice we made in isolation. First, the businesses running their payment flows through our platform won’t take reliability on faith — having a credible, tested DR story, not a slide with a promise on it, is part of what earns that trust. Second, as a platform building towards fintech certifications, region-level DR was a requirement we needed to get ahead of. This is a reflection on what it actually took to build it.

Evaluating the Models: Choosing Our Approach

There are three broadly accepted DR models, and we evaluated all three seriously before committing.

Backup & Restore — and why it did not fit our payment path

Backup & Restore is the baseline. You periodically snapshot your data — RDS snapshots, S3 backups — and restore from them when disaster strikes. It’s simple and cheap, but recovery is measured in hours. For a general-purpose web application, that might be acceptable. For a payments system with strict settlement windows and regulatory obligations, an RTO of several hours and an RPO that could mean losing a meaningful slice of transaction data is not a conversation you can have with the business. We ruled this out early.

Active-Active — The Complexity We Weren’t Ready For

Active-Active is often treated as the ideal availability model on paper. Both regions serve live traffic simultaneously. Failover is seamless — no warm-up, no DNS switch lag. But the engineering complexity is significant, and for our stack it ran into concrete walls at the data layer.

Aurora Global Database supports only a single writer region. To achieve true Active-Active with Aurora, you’d need application-level conflict resolution for concurrent writes, or a move to a distributed database like CockroachDB or DynamoDB Global Tables — neither of which was a path we were willing to take mid-build. For Kafka, Active-Active means producers and consumers running simultaneously in both regions. MirrorMaker 2 does support active-active cluster mirroring, but consumer group offset synchronization across regions is eventually consistent. Managing exactly-once semantics for payment events in that setup is a significant engineering problem in its own right.

Beyond the data layer complexity, payments transactions were latency-sensitive. Synchronous cross-region writes to keep both regions consistent would introduce latency we weren’t willing to absorb on every transaction. Active-Active was the aspirational answer, not the practical one for where we were.

Active-Passive — The Pragmatic Middle Ground

Active-Passive gave us the right trade-off. One region handled all production traffic. The second region stayed warm — infrastructure provisioned, data replicated near-real-time, ready to receive traffic — but sat idle under normal conditions. No cross-region write latency on the transaction path. All writes went to the primary region; replication to DR was asynchronous and didn’t touch transaction latency. Recovery required a deliberate failover, but with the right setup, RTO was bounded and RPO was tight enough to meet our obligations.

Our RPO requirement was tight but not zero — Active-Passive with continuous replication could meet it. Our RTO requirement was in the range where a manual-but-fast failover procedure — measured in minutes, not hours — was viable. A tested runbook beats an untested automation, especially the first time you build this. We committed, and scoped the work.

Under the Hood: How We Built the DR Stack

Network Layer

We started with networking because everything else depends on it. Getting this wrong means your replication paths, your private service communication, and your failover routing are all built on a shaky foundation.

Cross-Region Connectivity

We used AWS Transit Gateway to establish private connectivity between the active and passive VPCs. This gave replication traffic (DMS, MirrorMaker) a private path that doesn’t traverse the public internet, which mattered for both latency and security.

VPC Design

Both regions followed the same VPC structure — same subnet layout, same CIDR conventions, same security group naming. This was a deliberate decision that made IaC reuse straightforward and reduced cognitive overhead when debugging cross-region issues. Diverging VPC designs was a debt that would compound fast.

DNS and Failover Routing

We used Route 53 to manage DNS. In a failover event, the DNS record for our primary domain was updated to point at the DR region’s API Gateway endpoint. TTLs were set low enough that DNS propagation didn’t become its own RTO contributor.

API Gateway and TLS

The passive region needed a fully operational API Gateway — not something spun up at failover time, but something that existed, was configured identically to the active region, and was ready to receive traffic the moment DNS was updated.

We provisioned the DR API Gateway as part of our standard IaC. Same routes, same integrations, same authorizers. TLS certificates for our domains were provisioned in both regions via AWS ACM. Certificate parity was verified as part of our DR drill checklist — a mismatched or expired cert in the DR region during an actual failover would be a bad day.

The DR Gateway sat behind its own load balancer. No traffic was routed to it under normal conditions — the switch happened at DNS.

The Data Layer

Relational DB: Aurora Global DB vs Replication through DMS

Our primary datastore was Aurora PostgreSQL. The first approach we evaluated was Aurora Global Database — AWS’s native cross-region replication for Aurora. It was elegant: sub-second replication lag, native promotion to a primary on failover. We tested it.

The deciding factor was cost. Aurora Global Database required us to use a supported memory-optimized instance class. For a platform still scaling up on the active side, and a passive region intentionally sitting idle, that baseline cost was harder to justify compared to a smaller DMS-based replication setup.

DMS gave us more control as well: replication tasks are configurable at the table level, transformation rules let us shape what gets replicated and how, and CloudWatch metrics give clear visibility into replication state. But it comes with real limitations. A few examples engineering teams should understand before committing to DMS:

• A captured table must have a primary key — tables without one are excluded from CDC.
• TRUNCATE operations are not captured — only INSERT, UPDATE, and DELETE are replicated.
• Tables with a unique index created using a COALESCE function are not supported.
• Column nullability changes are not replicated through change processing.

These are manageable constraints, but they need to be understood upfront — not discovered mid-replication. The full list is documented in the AWS DMS PostgreSQL source documentation.

The trade-off was operational overhead — DMS replication tasks needed monitoring, and failures surface differently than a native Aurora replica. For a passive region that was not serving reads under normal conditions, that operational overhead was easier to justify than the higher always-on cost floor of Aurora Global Database.

Queues: MSK and the MirrorMaker Setup

Kafka (via AWS MSK) was how our services communicated asynchronously — event publishing, payment state transitions, downstream consumer triggers. Replicating a Kafka cluster cross-region is a non-trivial problem because Kafka topics aren’t a database you can snapshot; they’re a stream of in-flight messages with consumer group offsets that carry meaning.

We used MirrorMaker 2 (MM2), Kafka’s built-in cross-cluster replication tool, running as a connector in the DR region’s MSK cluster. MM2 continuously mirrored topic data from the active cluster to the DR cluster. It also handled consumer group offset translation — mapping the active cluster’s offsets to equivalent positions in the DR cluster, which was what made it possible to resume consumption from the right position after a failover.

One important nuance: MM2 prefixes mirrored topics with the source cluster alias (e.g., primary.payments.events). Application code needed to know which topic name convention to use depending on which region is active. We handled this through configuration, not hardcoded topic names.

Object Storage: S3 Cross-Region Replication

S3 served two purposes in our setup — a document store for critical application data and a storage layer for frontend dashboard assets. Both needed to be available in the DR region.

We enabled S3 Cross-Region Replication (CRR) on the relevant buckets. CRR continuously replicated objects to a destination bucket in the DR region as they are written, which meant no custom replication logic to maintain. For the document store, this was important — critical information uploaded in the active region needed to be accessible if we failed over. For the dashboard assets, it ensured the frontend could be served without any dependency on the active region.

One thing worth noting: CRR replicated new objects going forward. Existing objects at the time we enabled it were not automatically replicated — we did a one-time sync using the S3 batch replication job to backfill the DR bucket before enabling CRR.

Secrets Manager: Keeping Credentials in Sync

An easy thing to miss: your DR environment needs all the same secrets — DB credentials, API keys, service tokens — and they need to stay current. AWS Secrets Manager supports native cross-region replication, which we used for most secrets. Any update or rotation in the active region was automatically reflected in the DR region.

Not everything was replicated though. Region-specific secrets — like the Kafka consumer feature flag — were intentionally kept separate per region. This flag controlled whether message processing was active or suppressed in a given region, so replicating it cross-region would have defeated its purpose.

Application Layer

Disabling Kafka Message Processing in the Passive Region

This was one of the more important design decisions at the application layer. In the passive region, Kafka consumers were running — they’re connected to the DR MSK cluster and the topics are being mirrored. But we didn’t want them actually processing messages. Processing in both regions simultaneously would have meant double-execution of payment logic, which is clearly unacceptable.

We controlled this with a feature flag stored in Secrets Manager, scoped to the region. On startup, each consumer checked this flag. If the region was marked passive, the consumer connected but suppressed processing — it read the flag value, skipped message execution, and logged that it was in standby mode. When we flipped the flag, consumers in the DR region started processing on their next poll cycle. No redeploy was required.

Externalizing All Region-Specific Configuration

Every endpoint, every topic name, every service URL was externalized into configuration — nothing was hardcoded. Services read their configuration from environment variables, which were populated from Secrets Manager at startup. This meant the same container image ran in both regions; behavior was determined entirely by configuration, not by what’s baked into the build.

Observability Layer

A DR setup was only as good as how current the DR data was. We monitored replication lag continuously for both data layers.

DMS

We tracked CDCLatencySource and CDCLatencyTarget via CloudWatch — the former flagged issues with DMS reading from the source, the latter pointed to write pressure on the DR Aurora instance.

MirrorMaker

We tracked records-lag per topic partition aggregated across all partitions, and alerted when it crossed a threshold that would put us outside our RPO window.

Both fed into a single DR health dashboard. Any sustained breach triggered an investigation — not a failover, but a prompt to find what changed. Replication lag ignored has a way of becoming an RPO violation at the worst possible moment.

After the Build: Drills, Drift and Staying Ready

The Drift Problem

Production infrastructure evolved constantly. New services got added. Security groups changed. Environment variables got tuned. Config parameters shifted. If DR infrastructure isn’t updated in lockstep, you accumulated drift — and drift meant your DR environment was a snapshot of what production looked like months ago.

We made it a process requirement: any infrastructure change to the active region required a corresponding change to the DR region in the same pull request. IaC (Terraform) made this tractable — DR is a separate workspace with its own state, driven by the same modules. But the process discipline mattered as much as the tooling. Drift didn’t announce itself; it was discovered at the worst possible time.

Drill Cadence

We scheduled formal DR drills annually, with smaller validation checks happening as part of regular infrastructure and release changes. The goal was not to demonstrate that everything worked — it was to find what didn’t before a real event forced us to find out.

Each drill followed the actual SOP, run against a staging environment that mirrors the production DR setup. We verified:

• Data sanity: Were the expected records present in the DR Aurora instance? Were counts consistent with the active region within expected replication lag?
• Config parity: Did the DR environment have all the secrets, environment variables, and configuration values that the active environment had?
• Replication lag at drill time: What was the actual lag across DMS and MM2? Was it within our RPO bounds?
• Consumer behavior: Did Kafka consumers in the DR region correctly start processing after the flag flip? Did they resume from the right offsets?
• End-to-end transaction flow: Could a payment be initiated, processed, and settled entirely through the DR region?

The Failover Runbook

When we declare a DR event, the sequence was straightforward but had to be executed in order.

First, we disabled Kafka processing in the active region by setting the dr-region-active flag to false in the active region’s Secrets Manager. Once confirmed, we flipped the same flag to true in the DR region, signalling consumers there to begin processing. Consumers picked up the new value on their next polling cycle, typically within 60–90 seconds.

Second, we updated the Route 53 DNS record to point at the DR region’s API Gateway. With TTLs pre-configured to be low, propagation was fast.

Third, we validated — test transactions, error rates, DB write traffic hitting the promoted Aurora instance.

Rollback early in the process meant reversing the flag flip and DNS change. After significant writes had landed in DR, rollback became the failback problem.

What We Deferred, and Why

The Failback Problem

We built the failover path. The road back — returning to the active region after a DR event — was deliberately left for later. The core reasoning was that failback doesn’t carry the same urgency as failover. Once you’ve successfully failed over, the platform is stable and serving traffic from the DR region. There’s no strict RTO on the return journey — it’s something you can plan and execute carefully post-disaster, without the pressure of an ongoing outage.

That said, failback is genuinely harder than failover. By the time you’ve been running on the DR region for hours or days, new data exists there that doesn’t exist in the primary. You need to reverse replication — stand up DMS in the opposite direction, let the primary Aurora catch up, reconcile any state divergence. For Kafka, you need to ensure consumer offsets on the return path are correct and that no messages are double-processed or dropped. The order of operations matters: replication first, validation second, DNS last — same as failover, but in reverse and with a database that’s been accumulating writes in the “wrong” place.

It’s a problem worth solving, but one you can afford to solve with time on your side.

Automated DR Declaration

The decision to invoke DR was a manual one. There was no automated trigger that detected a regional failure and initiated the failover sequence. Route 53 was in place for DNS management, but there were no health checks configured to trigger automated failover — the DNS update and the Secrets Manager flag flip were both manual steps.

This meant our RTO had a human response time component baked in — someone had to be paged, assess the situation, and make the call. Automating this safely is non-trivial: you need to be confident the trigger condition is a genuine regional failure and not a monitoring blip, a network hiccup, or a cascading alert storm. False-positive automated failovers in a payments system carry their own risks.

We hadn’t automated this by the time the project wrapped. It’s a known gap.

Closing Thoughts

DR is often framed as an infrastructure concern. In practice, it reached into every layer — network topology, data replication, application behavior, and operational process.

Active-Passive gave us a practical path to region-level recovery without adding cross-region latency to every payment transaction. It was not the most sophisticated architecture possible, but it matched the reliability goals, cost constraints, and operational maturity we had at that stage.

The system was not perfect. Failback and automated DR declaration were still open problems. But the important shift was that DR moved from being an assumption — and a compliance checkbox — to being a tested capability.

For a fintech platform, that matters. The value of DR is not just having another region; it is knowing how the system will recover when the primary path is no longer available.

Serverless architecture was not new to me conceptually, but my real-world exposure started when I got the opportunity to work with a team that was already using Lambda as part of its deployment stack.

The initial idea made sense. Lambda was used for event-driven workloads where running an always-on compute did not feel necessary. For those workloads, the value proposition was strong: process work only when there is work to process, scale without managing a worker fleet, and avoid paying for idle capacity. That model worked well for the use cases it was originally meant for.

The challenge started when the same pattern expanded beyond its natural boundary. Lambda was no longer used only for event-driven processing. It had also become part of latency-sensitive paths, including APIs and upstream services that other APIs depended on.

The issue was not Lambda itself. The issue was using the same execution model for workloads with very different latency and throughput expectations.

When I started working on the system, the goal was to improve throughput and reduce end-to-end latency. That required looking beyond individual functions and understanding the full request flow: which parts truly benefited from Lambda’s elasticity, and which parts needed a more predictable compute model.

Where Lambda Started Hurting in Production

The issues started showing up when Lambda moved beyond the workloads it was originally suited for.

It was no longer used only for event-driven processing. Some latency-sensitive APIs had started depending on upstream services that were also deployed on Lambda. That made the end-to-end request path harder to predict.

As Lambda-backed dependencies increased in the request path, latency predictability came down. Cold starts, initialization time, VPC networking, and downstream calls each added their own variance to the end-to-end flow.

We were using an execution model designed for elasticity in places where the system needed predictability.

Cold starts and execution lifecycle

Cold starts were one of the main latency contributors for latency-sensitive APIs.

When a new execution environment is needed, AWS has to prepare the runtime, load the code, initialize dependencies, and then invoke the handler. If the environment is warm, that cost may not be noticeable. If it is cold, that initialization becomes part of the request path.

For asynchronous processing, this was usually acceptable. For latency-sensitive APIs, it was not.

The harder cases were low-QPS and spiky-traffic APIs. With low traffic, execution environments may not stay warm. With sudden spikes, Lambda may need to create additional execution environments quickly. In both cases, requests are more likely to pay the cold-start cost. The cold start cost became significant when it entered the user-facing request path.

We looked at the usual mitigations. Periodic warming can help, but it was not deterministic. AWS can still recycle execution environments. Warming one environment does not guarantee that all future requests will be warm. If concurrency increases, new environments can still be cold.

Provisioned concurrency is more reliable because it keeps execution environments initialized and ready. But it changes the cost model. We are now paying for readiness, not just execution.

It also expects more capacity planning. Provisioned concurrency can be autoscaled using Application Auto Scaling, but it has to be explicitly configured and tuned. In our case, we had not enabled that auto-scaling path, so the configured provisioned concurrency became another number we had to reason about upfront.

If we under-provisioned, requests could still spill over to cold environments. If we over-provisioned, we paid for idle warm capacity. Provisioned concurrency reduced cold starts, but it also moved Lambda closer to the world of pre-provisioned capacity.

Application concurrency model

One thing Lambda simplified was the application concurrency model.

A single Lambda execution environment handles one invocation at a time. When concurrency increases, Lambda scales by creating or reusing more execution environments, not by running multiple invocations concurrently in the same process.

That means the handler usually does not need to be written like a multi-threaded web service that handles many requests inside the same runtime. For many workloads, this makes the application code simpler.

But this model also has a trade-off.

Since concurrency is achieved by creating more execution environments, each environment may have its own initialized runtime, cached clients, open connections, and memory footprint. This is different from a service running on ECS or EKS, where we can reason about concurrency within a smaller number of long-running processes.

This becomes critical when the function talks to downstream systems such as databases, caches, or internal APIs.

For example, if concurrency increases quickly, Lambda may create many execution environments. Each environment may open its own database connection or client connection. The function scales, but the downstream dependency may not scale at the same rate.

Lambda simplifies in-process concurrency, but it shifts the scaling concern to execution environments and downstream dependencies.

Execution environment reuse also needs care. Since Lambda does not run multiple invocations concurrently in the same execution environment, the concern was not in-process parallelism. The concern was execution environment reuse: stale state, cached data, temporary files, or open connections carrying into a later invocation.

We do not usually manage thread-level concurrency inside one process. But we still need to manage concurrency at the system boundary.

Lambda in a VPC

VPC attachment was another area where Lambda needed clearer boundaries. Attaching Lambda to a VPC should be deliberate, not a default setting.

If the function needs access to private resources such as RDS, ElastiCache, private internal APIs, or services in a restricted security zone, VPC attachment makes sense. This is especially relevant for workloads that process sensitive data, PII, financial data, or red-zone services where network isolation is part of the security model.

But VPC attachment is not free. It adds networking concerns to a compute model that is otherwise simple. Subnets, security groups, route tables, NAT gateways, VPC endpoints, IP availability, and debugging network failures become part of the Lambda operational surface.

AWS has improved Lambda VPC networking significantly over time, so the old blanket statement that Lambda in a VPC always has bad cold starts is no longer accurate.

But VPC configuration still adds architectural complexity and can affect startup behavior during scale-out. When concurrency increases quickly, subnet capacity, routing, security groups, NAT gateways, and VPC endpoints can all influence how reliably the function reaches its dependencies.

The guideline we followed was to make VPC attachment an exception, not the default. Use it when the function needs private resources or stronger isolation. Avoid it when the workload does not require that boundary.

Package and container size

Package size also became part of the latency conversation. In Lambda, dependency weight is not just a deployment concern. It can affect startup behavior.

Large ZIP files, heavy container images, unnecessary SDKs, expensive framework initialization, and eager startup logic all add work before the handler can process an event. Initializing clients outside the handler can help with warm reuse, but doing too much upfront can make cold starts worse. Anything initialized before the handler should justify its place in the startup path.

Dependency discipline had a bigger impact in Lambda than we initially expected. It was not just code hygiene. Every unnecessary dependency, heavy framework, or expensive initialization step could directly affect startup behavior.

A small deployment package and a large container image may run the same business logic, but they are not the same operational experience.

Observability and runtime readiness

Another place where Lambda needed more care was runtime observability.

In ECS or Kubernetes, we are used to readiness and liveness probes. A container can tell the platform whether it is alive and whether it is ready to receive traffic. In Lambda, there is no continuously running process to probe. The health boundary is the invocation.

That changed how readiness had to be understood.

A function was healthy only if real invocations were succeeding within the expected latency and error budget. For async workloads, that also meant looking beyond invocation success alone. Queue age, DLQ depth, retry behavior, throttles, and downstream dependency errors all became part of the health picture.

The useful signals were not just whether the function existed or deployed successfully. They were whether the function was processing events correctly, staying within timeout limits, avoiding unexpected cold-start impact, and not silently building backlog.

So observability had to be designed around runtime behavior:

• invocation success and error rate
• duration and timeout rate
• cold start visibility
• throttles
• DLQ depth
• queue age or iterator age
• downstream dependency failures
• per-version or per-alias metrics where relevant

Circuit breaking also became less straightforward. In a long-running service, an in-memory circuit breaker can maintain state across many requests in the same process. In Lambda, execution environments are isolated, and each warm environment may have its own local view of downstream health. When concurrency scales out, that local state does not represent the whole function fleet.

For critical dependencies, circuit breaking and backpressure need to be designed outside a single Lambda runtime, using external state, queues, reserved concurrency, API-level throttling, or downstream-aware limits.

For Lambda, runtime readiness is not proven by a probe. It is proven by invocation behavior.

Versioning, canary, and rollback

Versioning and rollout were another area where Lambda needed explicit design.

With services running on ECS or Kubernetes, rollout patterns are usually part of the standard operating model. We think about deployments, health checks, rolling updates, traffic shifting, rollback, and service-level metrics.

With Lambda, the primitives are different.

Lambda gives us versions and aliases. A version represents an immutable snapshot of the function code and configuration. An alias can point to a version, and weighted aliases can be used to shift a percentage of traffic to a newer version.

That gives us the building blocks for canary-style releases. But those building blocks are not the same as a complete release strategy. We still needed to define how versions were published, which alias represented production, how traffic would move to the new version, what metrics would decide whether the release was healthy, and how rollback would happen if something went wrong.

A successful Lambda deployment did not always mean the new version was production-ready. The image may be valid, the function may be updated, and the alias may point to the new version. But real invocations could still fail because of missing configuration, dependency initialization, downstream connectivity, or event-shape issues. That made release validation important.

A safer release flow had to include more than deploying the function. It needed test invocations with representative events, log and metric validation, alarms on errors and timeouts, and a rollback path tied to those signals.

For latency-sensitive paths, this became even more important. A bad release was not only a correctness issue. It could also introduce higher cold starts, slower initialization, dependency failures, or increased downstream latency.

Compared to Kubernetes or similar service platforms, Lambda has fewer built-in release-safety patterns around readiness checks, rollout health, and rollback. Those controls need to be designed through validation, metrics, alarms, and deployment automation.

Where Lambda Still Fit Naturally

The places where Lambda still made sense were the ones where the workload was already asynchronous, retry-friendly, or not business-critical from a latency standpoint.

SQS-based processing was a good example. No user was directly waiting on a specific invocation, a small cold start delay was usually tolerable, and failures could be retried or moved to a dead-letter queue.

It also made sense for APIs where latency was not business-critical. For example, an API that triggers a notification, accepts a webhook, starts a background workflow, or supports low-frequency internal automation may not need strict p95 or p99 latency guarantees. In those cases, occasional startup variance was acceptable.

For systems where observability, versioning, rollout safety, or failure handling were critical, Lambda still needed deliberate platform work. We had to account for invocation-level health, logs, metrics, alarms, aliases, canary rollout, rollback, and post-deployment validation.

That created another trade-off. Even if Lambda reduced execution cost, it could increase implementation and operational cost. If the team had to build too much surrounding machinery to make the function production-ready, the benefit of using Lambda became less obvious.

That became one of the key decisions during the redesign.

Lambda could stay where the workload was short-lived, asynchronous, retry-friendly, or latency-tolerant. But for request paths where APIs were waiting on upstream services, and latency directly affected the user experience, the same model became harder to justify.

The goal was not to remove Lambda everywhere. It was to use it where its execution model matched the workload.

Other Boundaries Worth Knowing

Apart from the issues we directly handled in the latency path, there were a few broader Lambda boundaries worth keeping in mind.

These were not always the immediate bottlenecks, but they still influenced where Lambda made sense and where another compute model was a better fit.

The execution-time boundary

Lambda has a maximum execution limit of 15 minutes.

That is enough for many short-lived workloads, but it is not a good fit for jobs that are naturally long-running or unpredictable in duration.

This matters for long-running consumers, heavy ETL jobs, large data processing flows, and workloads where execution time can occasionally exceed the limit.

The question is not:

Can this usually finish within 15 minutes?

The better question is:

Can this reliably finish within that boundary under production conditions?

If the answer is “usually, but sometimes it may take longer,” Lambda is probably not the cleanest fit. ECS, Fargate, AWS Batch, Step Functions, or EC2 may be better candidates depending on the workload.

The same caution applies when Lambda is used in synchronous API paths. The question is not whether Lambda can serve an API. It can. The question is whether that API can tolerate invocation-time variance or long-running work. If not, the work is usually better moved to an async path.

Streaming, WebSockets, and gRPC

Another boundary is the communication model.

Lambda works best when the interaction is request-in, process, and return. But some workloads need long-lived connections, bidirectional communication, or service-to-service streaming.

Lambda can support some of these patterns indirectly. For example, API Gateway WebSocket APIs can invoke Lambda for connection and message events. Lambda also supports response streaming for some use cases.

But the model is still invocation-oriented.

The persistent connection or streaming contract is not owned by a long-running Lambda process. That is different from running a service on ECS, EKS, or Fargate, where the process can own the connection lifecycle more directly.

For workloads that depend heavily on long-lived connections, bidirectional streaming, or gRPC-style service communication, Lambda may not be the most natural fit.

In those cases, a long-running service model is often easier to reason about.

Code portability

Another boundary is code portability. Lambda is not just a different deployment target. It has a different execution model.

Code written for Lambda often assumes handlers, event payloads, execution time limits, cold starts, execution environment reuse, and trigger-specific behavior.

Code written for ECS or EKS often assumes a continuously running process, readiness and liveness probes, stable in-memory state, background workers, HTTP routing, service discovery, and graceful shutdown.

Moving that code to Lambda, or moving Lambda-oriented code back to ECS/EKS, is not only a deployment change. It can become an application design change.

That is not a reason to avoid Lambda. But it is something worth being honest about:

Lambda can reduce some operational work, but it can also couple the application to Lambda’s invocation model.

The Real Boundary: Elasticity vs Predictability

After working through the latency and throughput issues, I started evaluating Lambda through a different lens: not as a cost efficient deployment option, but as an execution model with clear boundaries.

Lambda is useful when the workload benefits from elasticity. If the workload is event-driven, short-lived, retry-friendly, and tolerant of occasional startup variance, Lambda can be a good fit. It lets the system scale with demand without requiring a continuously running process.

But when the workload needs predictable latency, stable warm capacity, long-running execution, or reliable upstream response times, the decision changes. In those cases, I would evaluate EKS or another long-running compute model more seriously.

The cost model follows the same boundary. Lambda’s cost advantage is strongest when workloads are intermittent. Once we need provisioned concurrency or always-ready capacity, we are no longer paying only for execution. We are paying for predictability. At that point, the comparison with EKS becomes more important because capacity is more explicit and can be shared across services.

That became the real boundary for me.

The question was not whether Lambda was good or bad. The better question was whether the workload needed elasticity or predictability.

Lambda could stay in the elastic parts of the system. Latency-sensitive paths needed a more predictable execution model.

The redesign was not about moving away from Lambda. It was about using it where its execution model made sense.

I started working with Spanner in real production systems, not as a database I was evaluating.

I used it across fintech and payments platforms, where correctness, reliability, latency, and disaster recovery were part of the everyday engineering tradeoffs.

My first understanding of Spanner was that it is a Distributed SQL. I was initially drawn in by the familiar surface area: SQL, transactions, indexes, schemas, and many concepts I already understood from relational databases.

But the more time I spent working on it, the more I realized that the SQL interface abstracts the complexity, but it does not remove it. Understanding that distinction changed how I designed, operated, and reasoned about the system.

The Mental Model Shift

The deeper learning was that Spanner’s SQL interface does not remove the need to think about distribution.

In a traditional relational database, I could often reason primarily in terms of schema design, indexes, query plans, and transactions. With Spanner, those still mattered, but they were not enough.

I also had to start thinking in terms of Spanner’s design principles: primary keys and splits influenced data distribution, hotspots showed where write concentration could become a problem, distributed transactions made me think about which key ranges a transaction touched, and interleaved tables or indexes affected locality.

A simple example is primary key design.

In traditional relational databases, an auto-incrementing primary key is a common and often reasonable default. The storage engine can maintain ordered indexes efficiently, and for many workloads, this pattern works well.

In Spanner, the same instinct can be problematic.

Spanner stores rows ordered by primary key and distributes data across key ranges called splits. If the primary key is strictly increasing, every new write goes to the end of the key space. Under high write volume, writes can concentrate on a small key range instead of being spread across splits.

That is how a familiar relational pattern becomes a distributed systems problem.

The issue is not that auto-increment is universally bad. The issue is that in Spanner, primary key design directly influences data distribution. A key that is convenient for identity may still be poor for write distribution.

Similar nuances changed the design questions I asked. It was no longer just:

“Is this schema normalized?”
“Is this query indexed?”
“Is this transaction correct?”

It also became:

“Will this key design distribute writes well?”
“Can this access pattern create a hotspot?”
“Does this transaction touch related data or scattered ranges?”
“Will this index become a bottleneck under write load?”

These were no longer just database design questions. They became distributed systems design questions.

Designing for Spanner’s Distributed Model

That mental shift changed the way I looked at everyday database design choices.

The concepts were familiar: primary keys, indexes, transactions, schemas, and joins. But in Spanner, they had distributed systems implications.

A few areas started to matter much more than they would in a traditional relational database.

The Data Locality Problem

Data Locality was an area I had to think about deliberately. In a traditional relational database, parent-child relationships are mostly logical. We model the relationship, create indexes, and rely on the database engine to execute joins efficiently within the same database instance.

With Spanner, the question becomes slightly different. Because Spanner distributes data across key ranges called splits, related rows may not always live close to each other physically — they might be located in different nodes or splits. If a query frequently reads a parent row and multiple child rows, and those rows are spread across different splits, Spanner may need to coordinate distributed reads across nodes instead of serving the data from a colocated range.

That does not mean joins are bad in Spanner. But it does mean joins should be designed with access patterns in mind.

For example, in a payments system, an account and its related transactions are often read together. If those rows are modeled independently without thinking about locality, the read path may involve more distributed coordination than necessary.

This is where interleaved tables become useful.

Interleaving allows child rows to be physically colocated with the parent row. Instead of treating the relationship only as a logical foreign-key-style relationship, we also have to instruct Spanner that this data is commonly accessed together. That can make reads more efficient for stable, parent-centric access patterns.

But interleaving is also a commitment. If we interleave too aggressively, the schema can become tightly coupled. We may optimize one dominant access pattern while making other access patterns harder. Child tables that need to be queried independently, moved across ownership boundaries, or accessed without the parent may not be good candidates for interleaving. This matters when we have multiple subtables and joins.

The instinct may be to colocate everything that is related. But that is not always the right answer. If multiple child tables are frequently read together through the parent, interleaving can help. But if those child tables have independent query paths, different lifecycles, or different scaling characteristics, forcing them under the same parent can reduce flexibility.

So before interleaving, I would ask:

Is the parent the natural entry point for this query?
Are these child rows usually fetched with the parent?
Do the child rows share the same lifecycle as the parent?
Will this child table need to be queried independently at high volume?
Am I optimizing the common path, or overfitting the schema to one query?

The principle I would carry forward is:

Design locality around access patterns, not just table relationships.

In Spanner, related data does not automatically mean colocated data. Locality is something that needs to be designed intentionally.

Hotspots as a Design Signal

Hotspots often become a litmus test for how well the workload is designed for Spanner.

They can sometimes be attributed to sudden or unusual traffic patterns. But in my experience, that was rarely the case in production. More often, hotspots expose an access pattern, key design, index design, or aggregation strategy that needs another look.

They are one of the first visible signs that the workload is not distributing well. This becomes especially clear during load testing. A schema or access pattern that looks fine at normal traffic can start showing stress once write volume increases.

Sequential keys, timestamp-leading keys, high-frequency updates to the same entity, poorly distributed secondary indexes, tenant-level concentration, or synchronous aggregate updates can all concentrate traffic on a small part of the database. This is why hotspot prevention is not only about choosing a better primary key. It is also about understanding how the application sends traffic to the database.

With more production exposure, hotspotting became an early design signal for me, not just a production debugging problem. It is something I would actively look for during schema design, access-pattern reviews, and load tests.

The Cost of Unbounded Transactions

Keeping transactions bounded is good practice in any database. Traditional relational databases can also suffer when transactions are too large, too slow, or too contentious. But Spanner makes the cost of broad transactions more visible.

In a traditional relational database, a transaction that touches unrelated tables may still be expensive, but the problem may not surface as sharply because the coordination happens within a single database instance. With Spanner, the same transaction may need to coordinate across different key ranges, splits, or replicas depending on where the rows live.

That changes the cost model.

A transaction that updates closely related rows is very different from one that updates unrelated resources just because they are part of the same application workflow.

For example:

Transaction A:
- update account balance
- insert transaction record
- insert ledger entry

This may be reasonable if the rows are modeled around the same access pattern and locality.

But compare that with:

Transaction B:
- update account balance
- insert transaction record
- update merchant summary
- update user rewards
- update notification state
- update daily reporting aggregate

The second transaction may look convenient from an application point of view, but it asks Spanner to coordinate more unrelated data. Depending on locality, those rows may live across different key ranges or splits and that extra coordination can become a performance cost under load.

The point is not that Spanner transactions are fragile. The point is that distributed transactions need intentional design.

Use distributed transactions where correctness requires them, but avoid turning every workflow into one large database transaction. In Spanner, a transaction is not just a correctness boundary. It is also a locality and coordination decision.

The Hidden Cost of Indexes

Indexes are easy to think about only as read optimizations.

That is true in any database to some extent. Traditional relational databases also pay a write cost for maintaining indexes. Every insert, update, or delete may require index updates.

But in Spanner, there is an additional dimension: indexes are also distributed data structures. That means an index can introduce its own hotspot, even when the base table is well designed.

This was an easy problem to miss. A table’s primary key may distribute writes well, but a secondary index can still concentrate writes if its leading columns have poor distribution.

For example, an index created to support queries by creation time, status, or tenant may look harmless from a read perspective. But if new writes keep landing on the same part of the index key space, the index itself becomes part of the hotspot problem.

So with Spanner, I had to think about indexes in two ways:

Will this index make the query faster?
What write path does this index create?

That second question mattered more than I initially expected. The hidden cost of indexes is not just storage or write amplification. It is also the possibility of creating a new distributed bottleneck.

So I would not add indexes only because a query became slower. I would also ask:

Is this index supporting a critical access pattern?
Does the index key distribute writes well?
Can this index become hot under load?
Is this index worth the extra write cost?

Indexes are still useful. But in Spanner, they need the same distribution thinking as tables.

What Worked for Me

Despite the design complexity, there were parts of Spanner that worked well for me in production.

The familiar SQL experience helped adoption. Spanner does not force teams to abandon relational thinking completely. We are still operating with tables, schemas, indexes, queries, and transactions. For teams already comfortable with relational databases, that makes the system easier to approach.

Strong consistency and distributed transactions were valuable. In fintech and payments systems, transactional consistency and reliable state transitions matter more than convenience. Being able to rely on strong transactional semantics across distributed infrastructure is a major advantage.

Disaster recovery readiness stood out. Spanner made cross-zone and cross-region replicas easier to manage without having to operate the replication layer ourselves. There is a latency tradeoff, especially with multi-region configurations, but for use cases that require stronger DR, availability, or compliance posture, that tradeoff can be reasonable.

Where “Managed” Did Not Mean Effortless

Spanner handled many distributed database concerns for us, but there were areas where the abstraction was not enough on its own.

Split management was hard to reason about. Spanner manages splits automatically, which removes a lot of manual operational work. But the same abstraction also limits visibility. Because split management is handled internally by Spanner, it can be difficult for an engineer operating the system to reason about exactly how data is being split, moved, or adapted under load. Pre-splitting can help in known scenarios, but I see it more as a tactical option than a long-term design solution.

Observability had limits. Cloud Monitoring gives useful metrics, and Spanner provides signals around latency, CPU usage, query performance, locks, and hotspots. But deep debugging was still difficult. I often wanted to slice metrics by table, tenant, key range, query pattern, or traffic shape, and the default dashboards did not always make that easy. For a production payments system, high-level signals were useful, but they were not enough on their own.

CDC required more design than expected. Change Streams provide the primitive for capturing data changes, but they do not give a complete eventing architecture.

Compared with binlog or WAL-based CDC in MySQL and Postgres, Spanner Change Streams felt different to reason about. In traditional databases, CDC is commonly built by asynchronously consuming the database log. The log itself is part of the database’s durability path, but downstream CDC consumers are not part of the application transaction.

With Spanner Change Streams, downstream consumption is also asynchronous, but the change record is captured transactionally with the write. That reliability is useful, but it also means Change Streams should not be treated as a zero-cost eventing layer.

This matters more when multiple change streams watch the same high-write tables or columns. In those cases, a single application write may need to produce change records for more than one stream, which can add overhead to the write path.

That does not mean multiple change streams are always a problem. But they need to be designed intentionally, especially when the database has high write volume or when different consumers ask for overlapping streams.

The downstream side also needs its own design. Ordering, retries, replay, backfills, deduplication, lag monitoring, and failure handling still need to be thought through.

If the architecture depends heavily on events from the database layer, this complexity needs to be planned upfront.

Local development and integration testing had productivity friction. The Spanner emulator helps with basic development, but it does not fully capture the production behavior that matters for distributed workloads. Split behavior, contention, performance characteristics, and latency are hard to validate locally. For serious payment flows, we had to rely more on real Spanner environments for integration testing, which added cost and slowed down parts of the SDLC.

The pattern across these areas was similar: Spanner removed a lot of infrastructure work, but it did not remove the need for application-level design, instrumentation, and operational discipline.

A Note on CockroachDB as an Alternative

A natural alternative to Spanner is CockroachDB.

I have used CockroachDB in the past, but only at a limited depth compared with Spanner. So I would not present this as a deep hands-on comparison. My view is based on light usage, some architectural evaluation, and feedback from peers who have worked with it more closely.

At a high level, both Spanner and CockroachDB try to solve a similar class of problems: SQL, transactions, consistency, and horizontal scalability over distributed infrastructure.

The design patterns feel similar from a distance, but the implementations and operating models are different.

Spanner is deeply tied to Google Cloud and is fully managed. That can be an advantage if the rest of the system is already on GCP and the team wants to reduce database operational overhead.

CockroachDB is more cloud-agnostic. That makes it interesting for teams that care about portability, deployment flexibility, or avoiding deep coupling to one cloud provider.

The way I think about the choice is less about “which database is better” and more about the operating model a team wants.

If the team is already committed to GCP and values managed operations, Spanner can be a strong fit. If cloud portability, deployment control, or multi-cloud flexibility matters more, CockroachDB becomes a serious alternative to evaluate.

I would still treat the final decision as workload-specific. Both systems require distributed systems thinking, and both need careful design around keys, transactions, locality, and operational behavior.

Final Takeaway: Where Spanner Fits

I would treat Spanner as an intentional architecture choice, not a default database option.

It is not needed for every workload. For simple CRUD systems, moderate-scale applications, or teams that can solve the problem with Postgres or MySQL, Spanner may add more complexity than value.

Where it becomes interesting is in systems where transactional consistency needs to hold across distributed data, and availability and disaster recovery are core architectural requirements. At that point, the database is part of the platform’s consistency and resilience strategy.

The other important factor is team fit. Spanner works best when the team is comfortable designing with distribution in mind: key design, locality, hotspots, transaction shape, and indexes.

Given Spanner’s managed and proprietary nature, it may also feel limiting for teams that need deep visibility or control over database internals. Split management, data movement, and adaptation under load are intentionally abstracted away. That is useful operationally, but it can be harder when engineers need to reason precisely about how it behaves in different scenarios.

Vibe coding is no longer a novelty

By now, most engineering teams have seen some version of it: a developer gives an AI assistant a rough instruction, gets a working-looking implementation, runs it, pastes back the error, asks for a fix, and keeps moving forward one prompt at a time.

For quick experiments, this workflow is powerful. It reduces friction, removes the blank-page problem, and helps developers move from idea to output quickly.

The more I use AI across coding, debugging, refactoring, and review workflows, the more one pattern stands out: Generating code is easy; engineering a solution still requires ownership. This is where my use of AI started changing - from asking it to produce code quickly to using it as a partner in reasoning, designing, testing, and reviewing the solution.

Karpathy’s original description of vibe coding was intentionally casual: “fully give in to the vibes,” “forget that the code even exists,” accept changes, paste errors back, and keep moving until it mostly works.

That description feels accurate because it captures what many of us have experienced with AI coding tools: the speed, the flow, and the temptation to trust code we have not fully read.

But as AI coding moves from personal experiments into professional engineering workflows, the question changes from “can it generate code?” to “can this build a software we can safely ship and maintain?”

Karpathy has since suggested “agentic engineering” as a better term for this more serious mode of working.

That term resonates with me because it moves the focus from casually prompting for code to deliberately guiding AI through the engineering process.

Why Vibe Coding Clicked for Me

Vibe coding felt powerful because it lowered the friction of starting. I did not need to fully understand the whole problem upfront. I did not need to map every edge case before starting. I did not need to design the complete solution in my head. I could simply begin. It removed the hardest first step: deciding where to start — blank page problem.

I could prompt:

Build me a login page.
Add validation.
Fix this error
Make it responsive.
Connect it to the API.
The API response shape changed — update the code.
Now add loading and failure states.

This style felt empowering because I did not have to carry the full cognitive load of the problem at the beginning. The AI gave momentum. And momentum matters.

For prototypes, internal tools, proof of concepts, small scripts, learning exercises, and quick UI experiments, vibe coding can be incredibly useful. It helps move from idea to working output much faster than starting from a blank file.

That speed matters, especially when the task is clear and I need momentum. But speed was not the only reason vibe coding felt natural to me.

It also mapped almost perfectly to an old developer habit: search, try, fail, refine, repeat.

Even before AI assistants, I rarely solved a feature from one perfect search query. The implementation usually moved through a chain of searches and decisions:

How to build a login page in React?
React form validation example.
Fix controlled input warning in React.
Make login page responsive CSS.
Connect React form to API.
Handle API error response in React.
Show loading state during form submit.

That flow already looked similar to vibe coding. The difference is that earlier, I had to search, read, compare, adapt, and stitch the solution into my codebase myself. With AI, the same chain can happen inside a chat window, and the assistant can generate a more complete patch at each step. That makes the workflow feel familiar. It also makes it easier to skip the adaptation step where understanding usually happens.

And that is where the trade-off begins.

The Review Problem

The first place this trade-off shows up is code review.

AI can produce code that is clean, formatted, and project-shaped. It can look like something an engineer deliberately designed. It can also produce more code than we would normally write by hand in the same amount of time: larger diffs, more files, more helper methods, more abstractions, and more tests.

That changes the review burden. Code generation has become faster, but code review has not magically become easier. If anything, reviewers now have to inspect more code, understand more logic, and validate more assumptions.

The risk is not just that bad code gets written. The risk is that polished-looking code moves through review without the same depth of verification.

Sonar’s 2026 State of Code Developer Survey captures the contradiction clearly: most developers do not fully trust AI-generated code, yet less than half say they always check AI-assisted code before committing.

That is how review can quietly become shallow:

The reviewer skims the diff. The author assumes the AI got it right. The pipeline is green. The PR gets merged. But when the reviewer asks the author to explain why a certain piece of logic exists, the answer is sometimes:

That’s what the AI generated, and it looked right.

AI did not create this habit. It made the habit easier to hide. Before AI assistants, the explanation used to be:

I copied it from Stack Overflow and changed it a bit.

But there is an important difference. In the pre-AI world, copying from Stack Overflow still required some effort. Engineers had to understand the answer at some level, adapt it to their codebase, resolve integration issues, rename variables, match project conventions, and make the solution work in their specific environment. That adaptation process forced at least some level of ownership.

With AI coding assistants, the solution can arrive already shaped like project code. It may include naming conventions, framework-specific syntax, tests, and comments. Because it looks polished, it can create a false sense of correctness.

The risk is not that AI writes code.

The risk is that AI writes code that looks reasonable enough to avoid deeper review. That creates a new engineering risk: code that enters the system without enough human understanding behind it.

Stack Overflow’s Developer Survey shows the trust gap widening: trust in AI tool accuracy dropped from 43% in 2024 to 33% in 2025, while distrust rose to 46%. That skepticism is healthy. AI-generated code still needs the same review discipline as human-written code.

“That’s what the AI generated” should be treated the same way as “I copied it from the internet”. It may explain where the code came from. It does not justify why the code belongs in this codebase.

Once that code is merged, the organization owns it. The engineering team owns it.

Not the AI.
Not the coding assistant.
Not the conversation that produced it.

The Production Reality

The deeper trade-off with vibe coding is that the cost of not understanding the code usually arrives later.

It rarely shows up on the day the code is merged.

It shows up when there is a production issue at 2 AM.

It shows up when a customer hits an edge case that the prompt never mentioned and the review never caught.

It shows up when the original prompt is gone, the chat context is forgotten, and the generated code has become part of the system. At that point, nobody cares whether the code came from an AI or an engineer.

The only question then would be:

Who understands how this works, and who can fix it?

If the answer is “no one,” it is not just a code problem. It is a process failure. Code produced through vibe coding that nobody understands is still legacy code. It just becomes legacy code faster.

The Context Problem Behind Vibe Coding

So far, the problem may look mostly like a process issue. And at one level, it is. But there is also a deeper reason vibe coding creates this risk: the way LLMs work with context.

The challenge was not getting the model to write code. The challenge was getting it to work with the right context.

Vibe coding feels natural because I can move one prompt at a time. But over time, that also means the model is not always working from a clear, complete brief. In practice, the context slowly gets scattered across the conversation.

At that point, the AI is solving the problem one step at a time, based on whatever context is visible, recent, or emphasized in that part of the conversation. The risk is not simply that the model “forgets.” In many AI coding tools, previous messages may still be available within the context window. The bigger issue is that context becomes fragmented across a long conversation.

The latest prompt may emphasize one symptom while earlier design constraints, edge cases, or architectural decisions become less visible or less influential.

This creates a few challenges. The model may optimize for the immediate error instead of the overall design. The conversation may accumulate small changes that do not fully agree with each other. I may unintentionally provide incomplete context in later prompts. The generated solution may slowly become a chain of patches rather than a coherent implementation.

That is how a simple experiment can turn into code that nobody intentionally designed. The code works, but I may not fully own the reasoning behind it. The feature passes a happy-path test, but the design may not be scalable. The pull request looks complete, but the reasoning behind the implementation is still missing.

Vibe coding can produce code quickly, but without deliberate context and review, it can also produce code that is difficult to explain, maintain, or debug.

How Agentic Engineering Helps

Agentic engineering is not about making AI sound more advanced. It is about making AI-assisted development more systematic. Unlike vibe coding, where the conversation often moves one prompt at a time, agentic engineering starts by making the problem, context, constraints, and validation criteria explicit.

This does not mean writing a massive specification before every change. It means giving the AI enough structure to reason before it generates.

You clarify:

• What problem are we solving and why does it matter?
• What does the current system look like?
• What constraints should not be violated?
• What trade-offs should be considered?
• What should the AI not change?
• What are the expected tests?
• What needs human review before implementation?

This is not only about spec-driven development. SDD is one useful pattern, but the larger idea is to approach AI-assisted development with engineering discipline. The goal is to make the AI work from intent, context, constraints, and feedback loops instead of scattered prompts.

This is also the direction many newer AI development tools seem to be moving toward. Tools like AWS Kiro and Tessl are not just trying to generate code from prompts. They are trying to bring more structure around AI-assisted development through specs, context, validation, and repeatable workflows. That direction matters because the hard part is not always code generation. The hard part is making sure AI is working with the right intent, the right context, and the right validation loop.

The difference is visible in how the conversation starts.

Instead of asking:

Create an API to add a new customer.

A more agentic approach would first ask the AI to reason through the problem:

Problem
We need to create an API endpoint to add a new customer.

Goal
The endpoint should accept customer details, validate the input, store the customer record, and return a clear response to the client.

Context
The application already has an existing customer table and follows a service-layer pattern. Other APIs in the codebase use request validation, structured error responses, and audit logging.

Constraints
Do not bypass the existing service layer.
Do not store duplicate customers with the same email address.
Do not expose internal database errors directly to the client.

Ask
Before writing code, propose the API contract, validation rules, error cases, and implementation approach.

Output expected
Provide the endpoint design, request and response examples, files to change, tests to add, and edge cases that should be reviewed before merging.

This prompt is not the solution.It is a way to start the engineering conversation before code generation begins. The value is not that AI immediately produces the perfect implementation. The value is that AI is first asked to clarify assumptions, compare options, and make its plan reviewable. That shift makes the output easier to question, review, and improve before it becomes part of the codebase.

The goal is not to generate more code. The goal is to generate better-understood code. Agentic engineering gives AI a larger role in the development process, but it also gives the engineer a clearer role: define the intent, provide context, challenge the plan, validate the output, and own the final decision.

Vibe Coding vs Agentic Engineering

The difference is not that one is good and the other is bad. The difference is knowing when each mode is appropriate. Vibe coding is useful when the goal is exploration. Agentic engineering starts to matter when the goal is not just to make the code work, but to make it understandable, reviewable, and maintainable.

A Proposed Agentic Development Lifecycle

Agentic engineering works best when the engineer does not disappear from the process.

My recommendation is to think of it as a 10–80–10 model:

• 10% upfront planning by the engineer
• 80% AI-assisted implementation
• 10% human review, validation, and ownership

The exact percentages may vary by task, but the principle matters.

The engineer should spend enough time upfront to define the problem, constraints, architecture, and success criteria. Then AI can help with the heavy implementation work: scaffolding, coding, refactoring, tests, documentation, and iterations. Finally, the engineer comes back with focused review: checking assumptions, validating edge cases, reviewing trade-offs, and deciding whether the output belongs in the codebase.

The mistake is to skip the first and last 10%. That is when AI-assisted development becomes risky.

Here is the lifecycle I would suggest:

1. Start with the problem statement

Before asking for code, explain the problem clearly.

• What are we solving?
• Who is affected?
• What does success look like?

2. Provide system context

Give the AI relevant architecture, constraints, patterns, dependencies, and existing conventions. The better the context, the better the output.

3. Ask for options before implementation

Do not jump directly to code. Ask for two or three possible approaches with trade-offs.

4. Choose the design intentionally

AI can recommend, but the engineer should decide.

5. Generate an implementation plan

Ask for files to change, functions to update, tests to add, and risks to watch.

6. Let AI implement in controlled steps

Use AI for the heavy implementation work, but keep the changes small and reviewable. Avoid massive AI-generated diffs that are hard to inspect.

7. Review like a normal engineering change

Question the logic.
Check edge cases.
Verify assumptions.
Run tests.
Look for unnecessary complexity.

8. Ask AI to critique its own output

Use the model to identify possible bugs, missing validations, security concerns, or performance issues.

9. Document the reasoning

Capture why the chosen approach was used. Future maintainers need the reasoning, not just the code.

A quick review checklist

Before merging AI-assisted code, I would ask:

• Can the author explain the major logic decisions?
• Does the code follow existing architecture and conventions?
• Are edge cases covered beyond the happy path?
• Are the tests meaningful, or just generated to satisfy coverage?
• Is error handling explicit?
• Are security assumptions validated?
• Are new dependencies necessary and approved?
• Could another engineer maintain this six months later?
• Is the reasoning documented in the pull request?
• Has the AI output been treated as a draft rather than a final answer?

This checklist is not about slowing teams down.

It is about making sure speed does not silently turn into risk.

Conclusion

Does AI improve engineering productivity?

Yes.

But it should not come at the cost of generating code that no one in the organization understands.

Accepting AI-generated solutions without understanding the reasoning behind them will eventually create problems. Maybe not today. Maybe not in the first demo. But eventually, that code will need to be debugged, extended, reviewed, or explained.

If no one can clearly explain it, that is not productivity. That is deferred responsibility.

The real value of AI in software engineering comes when we use it for planning, scaffolding, refactoring, testing, documentation, and exploring solution options.

When AI supports solutioning and engineers still own the final design choices, code throughput can genuinely turn into productivity.

Vibe coding helps us move fast. Agentic engineering helps us move fast without losing ownership.

AI should make engineers more productive, not less accountable.

That is the shift worth making.

AdBlockers, the white knight of web surfers had always been the tormentor for webloggers. With the increase in agitation over the cluttered pop ups & banner ads across the web, the usage of adblockers has become mainstream among the web fanatics. Adblock users justify their act by claiming that it provides them a sophisticated browsing experience whereas webloggers curses, as it leeches their revenue. And we can’t deny that both of them have legitimate points to some degree. But if we have a closer look, it’s silently revolutionising the way of digital advertising.

Let’s start with the brief note on ads & ad blockers. Advertisements(both commercial and non-commercial) are marketing stunts to promote product/ service/ organisations by creating brand awareness or by providing info. It is not new to the business world and exists even before the invention of the internet. Advertisements are done through various medium like mass media such as newspaper ads, TV & Radio ads, internet such as email marketing & digital ads in blogs, publishers and outdoor campaign such as banner ads.

Now, what is ad blocker(Well, online ad blocker)? In layman’s term, it’s a tool to remove the unwanted elements/ads from the webpage to have a better browsing experience. To explain as/to a techy, it is a software/browser extension to block the third party javascript libraries from blacklisted domains that’ll load contents in the website which are mostly ads, social tracking widgets or data trackers. In case, the ads are loaded from the server itself like google or facebook, it’ll hide the contents. Some advance adblockers like and mobile operators like and are blocking the content even at network carrier level.

All this drama began with the invention of adblocker by Henrik Aasted Sørensen as a browser extension in 2002 for Phoenix (then, now firefox) but didn’t get popular at that time. However, over the past few years, there is a sudden in the acrimonic mindset over digital ads among the people and in the usage of adblockers, especially among young lads. If we list down the reasons for that, it can be mainly categorized into 3 types. The first and foremost reason is, some (**Attention** : not all :P) unscrupulous publishers/bloggers percept the audience as more of a product than as a customer. They put a lot of efforts to make the user click the ads rather than making the content better viewable which literally kills the user experience on the web. Another major concern is the violation of user privacy. Ad engines inject code & drop cookies through the widget to track the user behaviour & interests when they are just meant for displaying ads. Last but not least, it paves the way for breaching the security by allowing third party libraries to execute which could spread viruses to a large extent.

As per the by Pagefair and Adobe, it is estimated that ad blockers cost around ₹150 crores business to the publishers and have around 20 crores active users last year which increased by 41% when compared to previous year. As their primary revenue model gets affected, publishers are employing three different types of strategies to address the issue. Type 1, Engagement with customers( Saathvigam ) i.e. reaching out to the audience to disable ad blockers or whitelisting their site in adblockers by explaining them the impact of ad blocking. Type 2, Block the AdBlock Users( ) i.e. threatening ( ) the users to disable the adblocker by denying the access to the content. Type 3, Providing Alternative Choices ( No, it’s not prachotakam 😛 but the milder version of Bayanakam ) It is the amalgamation of Type 2 & subscription model i.e. you should either disable the adblocker or subscribe(Monetary subscription, not email ) to the content in the website. On analyzing the of these approaches, only 0.33% of the people has made the Adblock exemption on request, out of those, 33% eventually removed the exception. And for subscription model request, only 3 users per million who were given the option to make a donation did so. This represents less than $0.01 CPM which explains none of the approach yields the fruitful results.

Besides these values, even if you go through the process, except Type 1 all other approaches have been harsh on the users and were forcing them to come out of their comfort zone. Here, Publishers failed to understand, it’s not the war by users on them. Rather, it’s the war between the users and crooked websites who welcomes them with a bunch of popup ads and challenges them to find the legitimate download button among 10 others 😉 It’s against those amoral people, who least care about user’s basic privacy.

Advertisements have not been a problem for users but the way it is being implemented on the web is a concern for them. Yes! If it really provides the relevant/valuable content/ad in a non-intrusive manner, most of the people (at least me) won’t even think about using adblockers. The fact that Click through rate (CTR) of 44% for ads in the initial years declining to 0.06% in recent time evidently asserts that ads are not serving it’s purpose anymore. As John Oliver , there should be “clear line between Church and State”. Advertisements should be clearly tagged and separated from editorial content. And when it comes to relevant ads, instead of targeting personalized ads, ad engines could target contextual ads. For example, instead of throwing ads for flights from Delhi to Madurai in Stack Overflow site, if it comes up with career related ads or Amazon EC2/Azure instance ads which are relevant to existing content/people visiting that site, will also yields better result as personalized ads and could potentially solve the problem of privacy.

If the publishers/blogger thinks that they can make the user disable the adblocker by denying the access to the content, it is never going to happen. If they are going to use anti-adblock technology, the user gonna use anti-adblock killer . After all, it’s technology man! Finding a workaround for a technical problem is never going to be that much tough. And web surfers should also whitelist the websites that are not intrusive and provide pleasant browsing experience. It’s quite common to think, “ Disruption in a business is not a big deal, unless I have a problem over that 😉 “. But for the betterhood of the internet, both users and webloggers should encourage the efforts like acceptable ads and restore the parity in digital advertising 🙂

Above Charles Darwin’s quote(sidestepping this contrary) accords not only for Bio world but in Tech world also. In the history of tech world, technologies that have failed to adapt the changes, haven’t sustained for a long period of time. However, Web technology has adapted well enough to overcome a lot of challenges since its invention by Tim Berners-Lee in 1989. One such instance was on the arrival of full scale online applications like Outlook Web App, Google Maps in early 2000s, it struggled to compete with native desktop apps in terms of responsiveness as the browser was supposed to reload the page for every HTTP request that time. But with the introduction of AJAX in 2005, it has almost replaced most of the native desktop apps and established its supremacy.

As the tech world is migrating towards mobile now, web technologies are facing the challenge of survival again and its battle is against the native apps which defines the user experience for mobiles at present. Other than notification & background sync, the fundamental challenge mobile browsers faces, is accessing the app offline/slow internet connection . Some of the workarounds like AppCache, LocalStorage/IndexedDB that are employed to address this issue weren’t up to the mark. In addition to agony of development & debug issues for programmers, main reason those attempts failed to work was, they didn’t work in right layer ( Eh, what layer?)”for mobiles” (Make a note, for mobiles in quotes).

Before explaining about the layer, let’s start with later part, the word on quotes. Offline usecases for mobiles are more complex than in the desktop one. Arguably, we can articulate that network state in desktop is binary, either you have full network or no network. But, that’s not the case with mobile. There are a lot of possibilities that ranges from browsing at high speed wifi in office to browsing at BSNL 2G while travelling in electric train from Chromepet to Nungambakkam. For slower connections, it’s good enough to assume that the user is in offline but unfortunately earlier workarounds were unable to resolve that issue.

Coming back to earlier part, now it’s so clear that the problem is not in the caching layer alone. Athukkum mela! Yes, We need to take some decisions in network layer also. To do that magic, here comes the protagonist of this blogpost, “ServiceWorker”. It should be relatively new term for most of the people even for web pros. You would have firefoxed/chromed about it on reading the blog topic ( my atonements to the Grammar nazis for verbifying the browser names 😛). Because, it’s a new technology still in research stage.

So, What is this “ServiceWorker”? It’s a background script execute by the browser, isolated from web page’s process which act as a network proxy allowing developers to handle the network requests programmatically and decide the mode of response(either from cache or retrieve from online). To make it simple, it’s a layer between web page and network which can be handled by web developers (not ‘browser developer’). Since it is isolated from web page’s process, it’ll allow developers to push notification using Web Push API and perform background sync for the given page even if the web page is not active. To know more about the ServiceWorker, just go through the introduction video by Jake Archibald, Developer advocate of Google chrome.

Your’s concern is understandable. Why should the developers prefer this complicated( kind of) over the native mobile apps. Yes! I agree. Native mobile apps provide ample user experience at lesser complexity ( as of now). But the cost of making user to install & update the app is more than just entering url in the browser. And another main advantage web apps have, over native apps is its ability to perform in cross-platform.

The e-Commerce giant Flipkart which has failed in its attempt of app-only strategy , now in association with browser vendors (that provides low-level API’s based on the feed of developers) has launched it’s mobile app, Flipkart Lite using serviceWorker which can perform similar as its native app.

As a web developer, it sounds so delightful to know that browser vendors are breaking the limits of web platform in mobile by providing low-level APIs not to forget Extensible Web Manifesto ‘s efforts which made the feedback loop process even simpler. It’s too early to comment on how it’ll evolve but it succeeds in bringing the new dimension for solving the appification of web problem in mobile.

Hiding your personal data from others have always been a difficult job. Some people used to hide files by creating a hidden folder inside multi-level folders. Some others will do that by placing the files in between system files or any other better techniques they know. In this article, I’m going to share you a simple technique by which you can hide your data in a image. Most of the people on reading the topic/on seeing the featured image might think that I gonna share a Steganography/Cryptography technique. We gonna accomplish this by using the distinguishing feature of image and archive readers . Before explaining the technique, lemme start with how to do that

1. Select an image in which you want to hide the files. Say “myblog_image.jpg” .(It’s need not to be jpg file, it can even be of other formats like png,gif )
2. Compress the files(like videos, .exe, images, documents ) you want to hide, either to .zip/.7z/.tar.gz format. Say”myblog_compress.zip” ( It’s mandatory to compress the files to anyone of those formats. Reasons later )
3. Place both the files in a same folder. (It’s not mandatory.. You can have it in different folder also)

In Windows:

1. Go to Run(WinKey + R or Start->Run) and enter “cmd” or just use any easy way you know to open command prompt
2. Navigate to the directory where you have the image and compressed file.
3. Enter the following command.

copy /B <image_file_source>+<archive_file_source> <new_file_name>

Make sure you give it in a same order i.e. image source first and archive file source next (Reason later). In our case it’s,

copy /B myblog_image.jpg+myblog_compress.zip myblog_hidden_image.jpg

If you want to mention the file from different folder, give either relative or absolute path in double quotes.

copy /B “E:\Works\Blog\Hide Personal Files\myblog_image.jpg” + “E:\Works\Blog\HidePersonalFiles\myblog_compress_image.jpg” myblog_hidden_image.jpg

In Linux:

1. Open the Dash (Super Key) or Applications and enter terminal or click Ctrl + Alt + T to open Terminal (or any method you know to open terminal )
2. Navigate to the folder you have the image file and compressed file.
3. Enter the following command

cat [compressed_file_source] >> [image_file_source]

In our case,

cat myblog_compress.zip >> myblog_hidden_image.jpg

Unlike Windows, it won’t create a new image file. It’ll hide the zip in the same file itself.

If you open the image, it’ll display the image and there would be no trace for zip file in the image.

And if you change the extension to .zip and open in the archive manager it’ll open the files you archived
Moving to technical details, lemme give some brief sketch of info you need to know before explaining in detail.

(i) Each file is stored in a binary format.

(ii) Image formats like JPEG, PNG, GIF will store the size of the image, format and other details in the header whereas Archive formats like .zip, .rar store their data both in header and footer. And hence, image viewers reader reads the data from the header to EOF whereas Archive readers will read the data from rear side i.e. from footer to header.

Anyone who is good at windows commands/ linux commands would find that the above commands just combines the binaries of the image and archive file. i.e., Archive file binaries would be appended (not mixed) to the image binary file. The new file will look like,

So, while opening the file in image viewer, it’ll read from the image header and reads exactly the same number of bytes mentioned in the header and ignores the archive file. Similarly, when opening the file in archive viewer, it’ll reader the data from the archive footer and ignores the image file. Now you may come to know why i asked you to give the command in the order of image and archive The reason we are choosing archive files for appending is not only we can hide many files but mainly because of it’s viewers property of reading from rear side.

The only way the other people can find that you hide the data inside an image is by its size only i.e. if you have an image of size 500 MB, anyone with common sense will surely find it ? Otherwise you can’t find that, image has the binary of archive file too. If you want to add extra layer of security, make the password protected archive files In case of any doubts/clarifications, leave your comments below

PS: If you append two or more archive files to a image, you can access the last archived file only

Gone are the days when we need local guide during National/Overseas tour, subtitles/friend for other language movies and giving awkward look on seeing other language sign boards/banners , Thanks to Google Translate If you giggle, “Dude!! Grow up, its old story ? “, the answer in “I”s vikram style is, “Ippo Athukkum mela (Now it’s more than that) ? “

Yesterday Google released an important update to Android and iOS Google Translate App where you can translate the words sign boards/images in real time using Word lens by Quest Visual which it acquired on May,2014. The most important thing is it works without internet connection(Offline) tooo.. And as of now, it will work for limited languages only(English,French, German, Italian, Portuguese, Russian, Spanish) and ofcourse, other languages are on their deck.

Not only that, it also translates the sign boards, but also makes the bi-directional real time voice translation. What I mean by bi-directional here is if you select the two languages you gonna have conversation, it’ll automatically recognize the language during conversation i.e. you need not change the language manually ? Unlike Word lens, it works in data connection only.

And you might not forget that , Microsoft unveiled a preview of a similar service in English and Spanish that allows users for skype last month, to converse in real-time. The difference is that you can try that in desktop/laptops only whereas google go ahead a step by implementing it in smartphones. And it’ll be really interesting when it is implemented in Google wearables like Google glass

To speed up the system performance, I am not going to say, “Dudes, get high speed CPU, RAM, hard disk, etc.,” or “Install these X or Y software and speed up your system”. Similar to “Increasing Internet speed” article, I gonna share a windows technique which would increase your system performance to a extent

Before sharing the technique, let me give a glimpse on which factors speed of computers/laptops relies on. It depends upon hardware factors such as CPU Speed, Size of RAM, Hard disk rotation speed, Speed of Bus (Not MTC/SETC bus, it’s CPU bus which is used for communication of components in CPU). Malware such as Virus, Trojan can also affect the system performance.

The factor we gonna improve is the size of the RAM. Yes, in this article we gonna learn how to increase the size of the RAM You might have puzzled by this statement. “Increasing RAM size? So, he gonna share a hardware technique?”, might be your thought process now . No man!! We are going to increase the size of the RAM not physically but virtually ? Those OS freaks here would get what I am trying to say. Yes, We are going to increase Virtual memory for the system to improve its performance. For those people who haven’t heard/know about Virtual memory, lemme give a brief definition. A part of memory in the hard disk (Secondary memory) will be used as a primary memory (RAM) using some mapping techniques. To know more about Virtual memory, refer “Namma (our) Wikipedia “ ?

Moving to implementation, it can be accomplished in two ways.

(i) Using Command Prompt

1. Go to Run(WinKey + R or Start->Run) and enter “cmd” or any easy way you know to open Command Prompt <Make sure you open it as administrator >
2. Run the command “fsutil.exe behavior set memoryusage 2”
3. Reboot the system and check the performance in the system

(ii) Changing Registry files

Those who are allergic to command prompts can accomplish this by changing registry files

1. Go to Run(WinKey + R or Start->Run) and enter “regedit”.
2. Naviagate to “HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem” and double click the key “NtfsMemoryUsage”.
3. Set the value to 2 and reboot the system

Some of you will come up with questions like, “Bro, What if we set the memory usage with 3 or higher value ? Will it improve the system performance ? If so, what is the maximum value we can set for the memory usage?”. The answer is, “2 doesn’t represent the size of virtual memory. Its the option to increase the size of paged pool memory. ( What is paged pool memory and nonpaged pooled memory is a big topic to discuss here. If you are more interested in this topic, refer here ) The other options are 0 and 1 which means ‘Not Set’ and ‘Default’ respectively. If you assign the option ‘0’, it means you are not setting any value, which means System gonna take default value is ‘1’ ? To make it simple, the options ‘0’ and ‘1’ are nothing but the same And option 2 corresponds to increase the size of page pooled memory.

Now lets see in which cases this technique might fail ? Here, I want to quote Microsoft’s official documentation on this command,

Increasing physical memory does not always increase the amount of paged pool memory available to NTFS. Setting memoryusage to 2 raises the limit of paged pool memory. This might improve performance if your system is opening and closing many files in the same file set and is not already using large amounts of system memory for other applications or for cache memory. If your computer is already using large amounts of system memory for other applications or for cache memory, increasing the limit of NTFS paged and non-paged pool memory reduces the available pool memory for other processes. This might reduce overall system performance. This parameter updates the HKLM\SYSTEM\CurrentControlSet\Control\FileSystem\NtfsMemoryUsage registry key.

That is, it’ll improve performance if you work on same set of files not necessarily on same folder. To be precise, it’s nothing but cache of CPU. Less you change the cache file contents, higher will be its performance. So, this statement wouldn’t be a big deal for normal system users. And this is a great gift for gamers who would face “Sloooow System” problem For those who groans, “Dude, running above command decreases the performance of my system “, No issues buddy.. You can revert it to previous state just by running the command, “fsutil.exe behavior set memoryusage 1”. And for those who ask, “Bro, I dunno in which option is selected. How can i find the current selected option?”, you can find that by executing “fsutil.exe behavior query memoryusage”.

Feel free to comment down your queries/suggestions