The AI Migration Confidence Trap: When Generative AI Meets the Dunning–Kruger Effect

Why successful database modernization requires more than convincing AI-generated code

The Illusion of a Successful Migration

Generative AI is fundamentally changing how organizations approach software development, database engineering and technology modernization. Tasks that previously required hours of manual investigation can now be accelerated through Large Language Models (LLMs) capable of analyzing source code, translating SQL dialects, suggesting schema transformations and explaining unfamiliar database technologies.

For database migration initiatives, these capabilities present a significant opportunity.

Imagine asking an LLM to convert a Microsoft SQL Server stored procedure into Oracle PL/SQL. Within seconds, the model produces a seemingly complete implementation. The syntax appears correct, the procedural logic looks familiar and the accompanying explanation confidently describes the transformation.

The code may even compile successfully.

But does that mean the migration was successful?

Not necessarily.

Compilation establishes that the generated code satisfies certain syntactic and structural requirements. It does not establish that the resulting implementation preserves the original application’s behavior, transaction semantics, concurrency characteristics, numerical precision or performance expectations.

This distinction exposes a growing challenge in AI-assisted modernization: the gap between generating a plausible solution and possessing the expertise necessary to validate it.

That gap becomes particularly dangerous when combined with a well-known cognitive bias: the Dunning–Kruger effect.

Understanding the Dunning–Kruger Effect

The Dunning–Kruger effect, introduced through research by psychologists David Dunning and Justin Kruger in 1999, describes a relationship between competence and self-assessment in which individuals with limited skill in a particular domain may have difficulty accurately evaluating their own performance.

The underlying concern is not simply that inexperienced individuals can be overconfident.

It is that the knowledge required to perform a task competently often overlaps with the knowledge required to recognize mistakes in that task.

In database engineering, this distinction is especially relevant.

An engineer unfamiliar with database transaction isolation may successfully translate a stored procedure from one SQL dialect to another without recognizing that the target database handles concurrency differently.

A developer unfamiliar with query optimization may consider a migration complete because the converted SQL produces the expected results on a small development dataset.

An architect unfamiliar with distributed database behavior may accept an AI-generated recommendation without recognizing its implications for consistency, availability, or recovery.

In each situation, the absence of specialized knowledge makes it difficult to recognize the limitations of the proposed solution.

Generative AI introduces another dimension to this problem.

It can provide the appearance of expertise without transferring the underlying expertise to the person using it.

This does not mean that every inexperienced engineer will overestimate their abilities nor that the Dunning–Kruger effect explains every instance of misplaced confidence. Research into self-assessment and overconfidence is more nuanced than the familiar popularized confidence curve suggests.

However, the broader problem of inadequate self-evaluation is directly relevant to AI-assisted database migrations.

The New Confidence Trap: Plausibility Is Not Correctness

Traditional database migrations required engineers to understand substantial portions of both the source and target database platforms.

They needed to investigate data types, SQL dialects, procedural languages, transaction models, indexing strategies, optimizer behavior and operational requirements.

Generative AI can now produce much of the initial conversion work.

That is a genuine productivity improvement.

The challenge arises when organizations begin confusing the speed of code generation with the completeness of the migration process.

Consider the following progression:

1. An engineer submits a source database object to an LLM
2. The LLM generates a target database implementation
3. The implementation compiles successfully
4. A limited functional test returns the expected result
5. The engineer concludes that the object has been successfully migrated

At first glance, the process appears efficient.

But several important questions remain unanswered.

• Does the implementation preserve the original transaction boundaries?
• Are numeric precision and rounding behaviors equivalent?
• Are NULL handling and implicit type conversions consistent?
• Will the query perform acceptably against production-scale data?
• Are locking and concurrency characteristics preserved?
• Does exception handling maintain the intended application behavior?
• Are dependent objects and application integrations accounted for?
• Can the implementation meet recovery, availability and operational requirements?

An LLM may help answer these questions.

But it cannot establish their answers merely by generating code or asserting that its output is equivalent.

The danger is not that AI produces imperfect code. The danger is that imperfect code can appear sufficiently complete to discourage further investigation.

A Practical Example: Migrating SQL Server MONEY to Oracle AI Database

Consider a seemingly straightforward datatype conversion from Microsoft SQL Server to Oracle AI Database.

SQL Server provides a MONEY datatype that stores monetary values using four decimal places of scale.

An LLM might recommend converting this datatype to Oracle NUMBER(19,4).

For many workloads, that is a reasonable starting point.

-- Microsoft SQL Server
CREATE TABLE EmployeePayHistory (
    BusinessEntityID INT NOT NULL,
    RateChangeDate DATETIME NOT NULL,
    Rate MONEY NOT NULL,
    PayFrequency TINYINT NOT NULL,
    ModifiedDate DATETIME NOT NULL
        DEFAULT GETDATE()
);

A proposed Oracle implementation might look like this:

-- Oracle AI Database
CREATE TABLE EmployeePayHistory (
    BusinessEntityID NUMBER(10) NOT NULL,
    RateChangeDate TIMESTAMP NOT NULL,
    Rate NUMBER(19,4) NOT NULL,
    PayFrequency NUMBER(3) NOT NULL,
    ModifiedDate TIMESTAMP
        DEFAULT LOCALTIMESTAMP NOT NULL
);

The transformation looks reasonable.

The Oracle statement can be syntactically valid.

But several considerations remain.

Datatype Representation

SQL Server MONEY uses a fixed-scale representation, whereas Oracle NUMBER uses decimal numeric representation.

Although NUMBER(19,4) can represent the values in the SQL Server MONEY range, the representation and arithmetic behavior are not identical in every context.

Expression evaluation, intermediate results, rounding and implicit conversions require examination.

Temporal Semantics

SQL Server DATETIME and Oracle TIMESTAMP have different precision and representation characteristics.

Additionally, GETDATE() and LOCALTIMESTAMP are not exact semantic equivalents in every deployment configuration.

Time zone assumptions, session settings and application expectations must be considered.

Procedural Dependencies

A successful column conversion does not establish that stored procedures, functions, views and application code using the column have been migrated correctly.

For example, financial calculations may depend on implicit casts or rounding behavior that changes during conversion.

Performance and Indexing

Even if the data converts successfully, differences in statistics, index structures, query optimization and execution plans may affect workload performance.

The Engineering Conclusion

The LLM-generated mapping provides a useful initial implementation.

It does not provide sufficient evidence that the migrated workload is equivalent.

The correct next step is to validate the transformation using representative source values, arithmetic edge cases, application logic and production-relevant workload tests.

This is where engineering expertise remains essential.

Database Migration Is Not a Code Translation Exercise

One of the most persistent misconceptions surrounding heterogeneous database migrations is that their primary challenge involves translating SQL syntax.

SQL translation is certainly important.

But a database is more than a collection of SQL statements.

A production database represents a complex system of interconnected behaviors involving data, applications, transactions, security, performance, availability and operations.

Migrating that system requires understanding not only how the original implementation was constructed, but also why it behaves the way it does.

A useful way to examine the problem is through four migration disciplines.

1. Planning: Understanding What Must Be Migrated

Before generating conversion code, organizations must understand their source environment.

This includes database inventory, object dependencies, workload characteristics, application integration, compatibility requirements and business constraints.

LLMs can assist by summarizing metadata, explaining unfamiliar objects, identifying possible incompatibilities and suggesting assessment activities.

However, their conclusions depend on the information provided.

An LLM cannot reliably assess dependencies it has never been shown.

Incomplete assessment combined with confident AI-generated recommendations can create a false sense of migration readiness.

2. Preparation: Establishing a Valid Target Design

The target database implementation should reflect both source application requirements and the capabilities of the destination platform.

This often requires architectural decisions rather than literal translations.

For example, a migration from PostgreSQL or SQL Server to Oracle AI Database may present opportunities to reconsider indexing, partitioning, procedural logic, JSON processing, or application integration patterns.

An LLM can suggest alternatives.

But deciding among them requires knowledge of the workload, database capabilities, operational requirements and business objectives.

A syntactically faithful conversion is not necessarily an architecturally appropriate conversion.

3. Execution: Automating What Can Be Determined

Execution is where automation and generative AI can deliver substantial value.

Many migration activities are well suited to deterministic transformation rules.

Others require contextual reasoning because their implementation depends on application intent or platform-specific behavior.

The most effective approach is not to choose between deterministic automation and generative AI.

It is to combine them.

Use established conversion rules where transformations are well understood and repeatable.

Use LLMs to assist with complex cases that require interpretation, alternative implementations, or additional investigation.

Then subject the results to appropriate validation.

4. Validation: Proving That the Migration Works

Validation is the discipline that separates a converted implementation from a successful migration.

It must extend beyond checking whether database objects compile.

A comprehensive validation strategy should examine data fidelity, functional behavior, transactional correctness, application compatibility, performance, security and operational readiness.

Generative AI can help develop test cases, identify potential edge conditions and analyze failures.

But AI-generated tests can share the same incorrect assumptions as AI-generated code.

Consequently, validation must incorporate independent evidence from the source system, authoritative requirements and representative workloads.

AI should accelerate the creation of evidence not substitute its confidence for evidence.

The Risk of AI Validating AI

An especially concerning pattern emerges when the same generative AI workflow is responsible for both producing a migration and declaring that migration successful.

Consider the following sequence:

An LLM converts a stored procedure.

The engineer then asks the same model to review the converted procedure for correctness.

The model evaluates the code, provides a positive assessment and identifies no significant concerns.

The engineer interprets that response as independent validation.

But the review may simply reproduce the assumptions embedded in the original conversion.

Even using a different model does not automatically create independent verification. Multiple models can share training-data biases, reasoning limitations, or misunderstandings of database behavior.

This creates a form of circular assurance.

The system responsible for generating the proposed solution becomes the principal authority for determining whether the solution is correct.

A more defensible approach incorporates independent validation mechanisms, including:

• Deterministic schema and object comparisons
• Source-to-target data reconciliation
• Differential execution of equivalent test cases
• Boundary-value and regression testing
• Transactional and concurrency testing
• Performance testing using representative workloads
• Review of security, recovery and operational behavior
• Subject-matter expert review of high-risk transformations

LLMs can assist with these activities, but the evidence should not depend exclusively on an LLM’s interpretation of its own output.

A Better Architecture: Deterministic Automation, AI Assistance and Evidence-Based Validation

The future of database migration should not be framed as humans versus AI.

Nor should organizations assume that increasing the autonomy of an AI system automatically improves migration quality.

Instead, migration platforms should combine complementary capabilities with clearly defined responsibilities.

The important architectural principle is that the mechanism proposing a transformation should not be the sole mechanism establishing its correctness.

This separation is particularly important when introducing agentic AI.

An AI agent may eventually coordinate metadata discovery, generate conversion candidates, execute tests, interpret failures and iterate on remediation.

Such a workflow can substantially reduce manual effort.

However, greater autonomy also increases the importance of defined constraints, observable execution, independent acceptance criteria and escalation mechanisms.

An agent that can repeatedly modify code until its own tests pass is not necessarily converging on a correct migration.

It may simply be converging on an implementation that satisfies an incomplete test suite.

From AI Confidence to Migration Confidence

A practical improvement is to stop treating AI-generated confidence as a proxy for migration readiness.

An LLM may describe a conversion as highly reliable.

But that statement should not be confused with a statistically calibrated probability of correctness or an independently verified assessment.

Migration confidence should instead be grounded in measurable evidence.

For example:

These dimensions should be evaluated independently.

A database migration that achieves complete structural conversion but fails critical transaction tests is not ready simply because its overall conversion percentage is high.

Similarly, an application that passes functional testing but cannot meet production latency requirements has not achieved operational equivalence.

Readiness assessments should reflect explicit acceptance criteria, evidence coverage and unresolved risk rather than a single opaque confidence score.

This is an important distinction for executives overseeing modernization programs.

Progress measures how much work has been completed. Readiness measures whether the resulting system can safely fulfill its intended purpose.

The two are not interchangeable.

The Human Factor: Expertise Becomes More Important, Not Less

Generative AI changes the economics of producing technical implementations.

It reduces the effort required to generate initial code, investigate unfamiliar syntax and explore alternative designs.

But lowering the barrier to producing a solution does not automatically lower the expertise required to evaluate that solution.

In fact, faster generation can increase the volume of work requiring review.

An experienced database engineer may recognize that a converted SQL statement introduces implicit datatype conversions that prevent efficient index access.

A less experienced engineer may see a query that returns the correct result and conclude that the migration is complete.

Both engineers can use the same LLM.

The difference lies in their ability to interrogate the output.

This suggests an important evolution in technical responsibilities.

As AI becomes more capable of generating implementations, engineering value increasingly shifts toward problem definition, architectural judgment, validation design, risk identification and operational accountability.

The objective should not be to eliminate expertise from migration projects.

It should be to make that expertise more productive.

Organizations should also resist equating a person’s willingness to use AI with their ability to supervise it. Effective supervision requires domain competence, awareness of model limitations and the willingness to challenge apparently convincing results.

Executive Implications: Speed Without Assurance Is Not Modernization

For technology leaders, the central challenge is not determining whether generative AI should be incorporated into migration programs.

It should be evaluated as a potentially valuable accelerator.

The more important question is how organizations ensure that acceleration does not compromise reliability.

Executives should distinguish between three outcomes:

Faster conversion: AI reduces the effort required to produce target database implementations.

Faster validation: Automation and AI reduce the effort required to establish that those implementations satisfy defined requirements.

Faster modernization: The organization achieves a production-ready target architecture with acceptable risk, performance and operational characteristics.

These outcomes are related, but they are not equivalent.

A project that reduces code conversion time by 80% while increasing downstream remediation, production incidents, or operational complexity may deliver little actual modernization value.

Likewise, an impressive demonstration of AI-generated SQL conversion provides limited evidence of enterprise migration readiness.

Technology leaders should therefore require migration programs to report not only conversion throughput but also validation coverage, unresolved incompatibilities, performance results and residual risk.

AI adoption should strengthen engineering governance rather than bypass it.

Conclusion: AI Can Generate the Answer. Expertise Determines Whether It Is Right.

Generative AI represents an important advancement in database modernization.

It can accelerate assessment, assist with complex transformations, reduce repetitive engineering work and make specialized technical knowledge more accessible.

Those benefits are real.

But so are the risks of misplaced confidence.

The Dunning–Kruger effect offers a useful reminder that evaluating technical correctness often requires the very expertise that inexperienced practitioners may lack.

When combined with generative AI’s ability to produce fluent, convincing explanations and implementations, that limitation can become difficult to recognize.

The answer is not to avoid AI.

It is to build migration practices that use AI appropriately, combine deterministic automation with expert judgment and establish correctness through independent evidence.

The most successful migration strategies will not be those that generate the most code with the fewest human interactions.

They will be those that minimize unnecessary effort while preserving the engineering discipline required to deliver reliable systems.

AI can accelerate a migration. It cannot make an unvalidated migration trustworthy.

And ultimately, the goal of database modernization is not simply to move workloads faster.

It is to move them correctly, confidently and with a measurable understanding of the risks involved.

References and Further Reading

1. Kruger, J., & Dunning, D. (1999). Unskilled and Unaware of It: How Difficulties in Recognizing One’s Own Incompetence Lead to Inflated Self-Assessments. Journal of Personality and Social Psychology, 77(6), 1121–1134. https://doi.org/10.1037/0022-3514.77.6.1121

2. Dunning, D. (2011). The Dunning–Kruger Effect: On Being Ignorant of One’s Own Ignorance. Advances in Experimental Social Psychology, 44, 247–296. https://doi.org/10.1016/B978-0-12-385522-0.00005-6

3. NIST. AI Risk Management Framework (AI RMF 1.0). https://doi.org/10.6028/NIST.AI.100-1

4. NIST. Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile. https://doi.org/10.6028/NIST.AI.600-1