Testing Memory Safety in C++: Should We Rely on AI or Run Sanitizers Ourselves?

Artificial intelligence can provide substantial assistance in testing C++ programs, from configuring the testing environment and writing tests to interpreting errors and proposing fixes. However, relying on it requires distinguishing between reading code and offering an assessment, and running actual tests that produce verifiable evidence.

If an AI model declares a program safe after reading its files, that is a potentially fallible assessment. If it builds the program with sanitizers, runs specific tests, and provides execution logs, it has performed testing that can be reviewed and repeated. Even that result does not mean the program is free of all defects.

What Do Sanitizers Detect?

Sanitizers add checks to a program during compilation and then monitor specific types of errors at runtime. They are detection and diagnostic tools: they do not automatically fix the code or provide a comprehensive security certification.

Each tool has a different scope. Running one does not replace the other checks a project may require.

Where Does AI Add Real Value?

An assistant with access to the project and the ability to run build tools can help select and configure appropriate checks, create tests for empty inputs, large sizes, and corrupted files, and then execute those tests and analyze their results.

It can be particularly useful when interpreting lengthy reports: Where did the invalid access occur? Where was the memory allocated? When was it freed? Is the defect in the function that crashed, or in an earlier mistake involving an object's lifetime?

AI can also write fuzz tests that explore many varied inputs. For example, libFuzzer uses execution coverage information to guide input generation and can run alongside ASan and UBSan to improve the chances of discovering defects in data processing.

These capabilities depend on the tools and permissions available to the assistant. Writing test commands in a conversation does not mean those commands were executed, and predicting their output is not equivalent to obtaining actual results.

Why Is “No Errors Found” Not Enough?

Suppose an image-reading program contains a defect that appears only when processing a truncated file with invalid dimensions. If we test only valid images, the tests may pass without revealing the problem.

Dynamic checks depend on what actually executes and on the types of errors the tool can detect. An absence of warnings therefore does not establish the safety of untested paths or rule out errors beyond the tool's scope.

The result should be interpreted precisely: The enabled checks detected no errors during these tests, in this environment.

Similarly, memory safety is not equivalent to complete security. A program may have no detected memory errors while still containing defects in authorization, authentication logic, or request validation.

Does Detecting Leaks Establish Memory Efficiency?

A program may retain large amounts of data in containers that remain reachable, even though the data is no longer needed. It may also copy data repeatedly, perform excessive allocations, or consume increasing amounts of memory under load without producing a conventional memory leak.

Evaluating memory efficiency therefore requires additional measurements. Massif measures heap memory usage and helps identify its sources, while DHAT helps analyze allocation patterns and inefficient memory use.

It is also essential to measure the performance of the production build separately. Sanitizers introduce execution and memory overhead, so measurements from an instrumented build do not represent the program's normal performance.

Can This Be Applied to Any C++ Program?

This approach can be applied to many projects, but no single configuration works for every program. Suitability depends on the operating system, processor, compiler, libraries, and build configuration.

Some tools may require more extensive preparation of dependencies. MSan, for example, requires appropriate instrumentation of the code and its dependencies to avoid unreliable results. Certain linking configurations also have limitations: Clang's ASan documentation states that fully static linking of executables is unsupported.

An assistant should therefore understand the project's environment before proposing generic settings and treating them as a complete testing solution.

Which Is Better: Running the Tools Yourself or Delegating to AI?

When the same tool runs with the same settings, inputs, and environment, its detection mechanism does not become more accurate simply because AI launched it. The potential value comes from improving tests, expanding coverage, and accelerating diagnosis.

Running tests directly gives developers clear control and allows them to apply their knowledge of the program's intended behavior. However, this approach can also suffer from oversights or limited coverage.

The approach I recommend is to run repeatable automated tests, use AI to help prepare them and analyze their output, and have developers review the results and proposed fixes.

That review must confirm that a fix addresses the underlying cause, and that a warning has not disappeared merely because a check was disabled or code was excluded from analysis. The input or scenario that exposed the defect should preferably become a permanent regression test. The source revision, build commands, tool settings, execution logs, and untested areas should also be documented.

Used this way, AI can reduce effort and improve the quality of testing. Confidence in a program's safety, however, rests on verifiable evidence, broad testing, and an understanding of its limitations. A reassuring statement from a model is not enough.