#11 Foundation and Architecture: Language Implementation Project Structure -> Milestone — Project Structure with First Experimental Language File

#11 Foundation and Architecture: Language Implementation Project Structure -> Milestone — Project Structure with First Experimental Language File

The culmination of architectural planning and toolchain setup is reached with this milestone: successfully compiling and running the first experimental source file in our new C-style programming language. This step validates our modular project structure, build system, dependency boundaries, and runtime logic. It also serves as a checkpoint that demonstrates the interpreter pipeline is connected end-to-end—from reading a source file to executing its semantics.

This section documents the essential components and design insights required to produce the first executable .lang program using our interpreter, implemented with modern C++20/23.

1. Context: What This Milestone Proves

By this milestone, we aim to:

• Ensure the entire interpreter pipeline (Lexer → Parser → Evaluator) is functional

• Verify that the project structure supports running user code from a file

• Confirm the runtime can process basic constructs like function calls and print statements

• Execute a small .lang program with correct output

• Enable a basic CLI interface or REPL to interpret input files

This milestone serves as a working proof-of-concept interpreter, even if only for a tiny language subset.

2. Project Structure Recap

Let’s revisit the C++ project structure, now populated with functional components:

/ForgeLang
├── CMakeLists.txt
├── /src
│   ├── /core           → SourceManager, diagnostics, utilities
│   ├── /lexer          → Tokenization logic
│   ├── /parser         → AST construction (Expr, Stmt)
│   ├── /runtime        → Evaluator, environment, built-ins
│   ├── /main           → Entry point, CLI handling
├── /examples
│   └── hello.lang      → First experimental language program
├── /tests              → Interpreter and syntax tests (optional)
├── /build              → CMake out-of-source directory

Each module compiles independently and links via modern target-based CMake using target_link_libraries().

3. Minimal Feature Set for First Program

The first source file (hello.lang) and the interpreter must support a minimal set of language features:

• Function definition and execution

• Return value from main()

• Built-in print() function

• Basic arithmetic or string literals

Example hello.lang:

fn main() -> int {
    print("Hello, world!");
    return 0;
}

This program:

• Defines a main function

• Invokes a built-in print operation

• Returns an integer value to indicate successful termination

4. Interpreter Pipeline Overview

Each stage of the interpreter must now function with working integration.

a. Lexer Module

• Tokenizes input source string into a list of Token structures

• Handles keywords, punctuation, literals, identifiers

• Reports lexical errors (if any)

b. Parser Module

• Converts token stream into an abstract syntax tree (AST)

• Recognizes function definitions, statements, expressions

• Associates source locations for debugging and error reporting

c. Runtime Module

• Resolves and invokes main()

• Registers built-in functions (like print)

• Manages environment, call frames, and execution stack

d. Main Executable (forge)

• Reads .lang file

• Passes it through lexer → parser → evaluator

• Returns appropriate exit code and forwards any runtime output

5. Implementation Detail: First Language Features in C++

Token Definition (Token.hpp):

enum class TokenType {
    Fn, Return, Identifier, StringLiteral,
    LParen, RParen, LBrace, RBrace,
    Arrow, Semicolon, Comma,
    IntegerLiteral, Print, EndOfFile
};
struct Token {
    TokenType type;
    std::string lexeme;
    SourceSpan span;
};

AST Nodes (Expr.hpp, Stmt.hpp):

struct CallExpr {
    std::string callee;
    std::vector<Expr> arguments;
};
struct PrintStmt {
    Expr value;
};
using Stmt = std::variant<PrintStmt, ReturnStmt, FunctionDecl>;

Built-in Function Handling (Runtime.hpp):

class Runtime {
public:
    void register_builtin(const std::string& name, NativeFunction fn);
    void run(const std::vector<Stmt>& program);
};
void builtin_print(const std::vector<Value>& args) {
    std::cout << args[0].as_string() << std::endl;
}

6. CLI Interface (main.cpp)

int main(int argc, char** argv) {
    if (argc < 2) {
        std::cerr << "Usage: forge <file.lang>" << std::endl;
        return 1;
    }
    std::string source = SourceManager::read_file(argv[1]);
    auto tokens = Lexer(source).tokenize();
    auto ast = Parser(tokens).parse_program();
    Runtime runtime;
    runtime.register_builtin("print", builtin_print);
    runtime.run(ast);
    return 0;
}

This minimal driver program enables direct interpretation from a file using the command:

./forge examples/hello.lang

7. Output and Success Criteria

When running the above command, you should see:

Hello, world!

And the program should exit with code 0.

Success at this stage means:

• CMake builds all modules correctly

• Runtime system is functional for minimal execution

• Diagnostic system reports errors if the .lang file is malformed

• The pipeline operates in isolation and composes together seamlessly

8. Testing and Next Steps

After achieving this milestone, you can begin expanding:

• Add support for variable declarations and assignments

• Introduce expression evaluation: math, logic, comparisons

• Develop .test files for automation

• Implement control flow: if, while, for

This milestone validates the project’s architecture and marks the transition from design to iterative language development.

Conclusion

This first milestone—executing a .lang file through a modern C++ interpreter—demonstrates the viability of the architecture, the correctness of module boundaries, and the effectiveness of C++20/23 features in building a clean, modular interpreter. It is not just symbolic, but foundational: every future feature builds atop this working skeleton.

The interpreter is now ready for feature expansion, testing infrastructure, error reporting refinement, and language evolution—all of which will be addressed in subsequent chapters.