Magister Catalyst
User's guide

Install the SDK

The Catalyst SDK

The proprietary Catalyst SDK is distributed as a versioned zip for macOS on Apple Silicon. Extract it anywhere, set CMAKE_PREFIX_PATH to that directory, and use find_package(Catalyst CONFIG REQUIRED). The archive includes mc headers, shared libraries and Mac frameworks, Cosmic, Microcosm, Nexus, Python and Swift bindings, runtime dependencies, documentation, examples, resources, and configuration. Its CMake package links your application to the supplied libraries.

See the SDK instructions for setup, framework integration, and application deployment. The minimum macOS version is recorded in distribution.json. Xcode or Command Line Tools supplies the platform SDK; Metal compilation also requires Xcode's Metal Toolchain. Linux support is coming soon.

The included Python package uses CPython's stable abi3 interface (3.9 or newer). Add the SDK's lib/python directory to PYTHONPATH. The Swift package in share/Catalyst/swift compiles its wrapper sources with Swift 5.9 or newer and links to the bundled native library. Both expose cvar, CMessenger, and CServer, with their support types. See the binding guide and SDK instructions for integration details.

The SDK includes the compilers and dependencies needed to build Catalyst applications. CMake configures the bundled headers and libraries automatically; no separate Boost or LLVM installation is needed. Standard C++ projects use Apple's selected compiler.

Configure your environment

Set CATALYST_SDK to the extracted SDK directory. Add its commands to PATH and set MC_HOME so applications can find its configuration and resources:

export CATALYST_SDK="$HOME/SDKs/Catalyst-1.0.0-macos-arm64"
export PATH="$CATALYST_SDK/bin:$PATH"
export MC_HOME="$CATALYST_SDK"

For GPU programs, install Xcode's Metal Toolchain if it is not already available:

xcodebuild -downloadComponent MetalToolchain
xcrun -sdk macosx metal --version

Build your application

To use an installed Catalyst SDK from a CMake project:

cmake_minimum_required(VERSION 3.24)
project(MyApplication LANGUAGES CXX)
find_package(Catalyst CONFIG REQUIRED)
add_executable(my_app main.cpp)
target_link_libraries(my_app PRIVATE Catalyst::Shared)
catalyst_debug_symbols(my_app)

Save this as CMakeLists.txt beside main.cpp, then configure and build your application:

cmake -S . -B build -DCMAKE_BUILD_TYPE=Debug \
  -DCMAKE_PREFIX_PATH="$CATALYST_SDK"
cmake --build build

Catalyst::Shared links the supplied core library and provides its headers and dependencies. Use this target for applications using cvar or other compiled framework facilities. Catalyst::Catalyst supplies just the headers and their dependencies for header-only use. The SDK provides shared libraries and Mac frameworks.

The C++ API uses namespace mc and headers such as <mc/cvar.h>.

The C binding header is <mc/CCatalyst.h>. Its runtime entry points include cCatalystABIVersion() and cCatalystDrain(). The Swift package uses the CCatalyst C module; Python applications import catalyst.

<mc/core.h> is a convenience header for common Catalyst facilities, including containers, parsing, threads, pools, queues, commands, and messaging. It also provides short type aliases such as i8, f8, and Strings, and makes the std and mc namespaces available without qualification. Include individual headers when you want to keep names qualified.

The SDK also provides <mc/pool.h>, <mc/queue.h>, <mc/CCommand.h>, <mc/CMessenger.h>, and <mc/CServer.h> individually. Pool, queue, and command APIs are header-only. Link messaging applications with Catalyst::Shared.

<mc/image.h> provides JPEG encoding/decoding and pixel-buffer placement. These functions belong to libcatalyst and are available independently of the macOS drawing library.

CDB requires PostgreSQL 15 or newer; PostgreSQL runs as a separate service. Install a server locally for development, or connect to an existing PostgreSQL server. To start a new local service and create an empty example database:

brew install postgresql@18
brew services start postgresql@18
"$(brew --prefix postgresql@18)/bin/createdb" catalyst_demo

If PostgreSQL already runs on your machine, use that service or configure a different port for the new one. These examples use a new database named catalyst_demo.

Run the tutorials

The SDK's examples/ directory contains 33 commented Cosmic tutorials. Each program checks its results, including when built with optimization. Configure the supplied CMake project, then build and run the lessons:

cmake -S "$CATALYST_SDK/examples" -B examples-build \
  -DCMAKE_PREFIX_PATH="$CATALYST_SDK"
cmake --build examples-build --target catalyst_examples
./examples-build/example-values
./examples-build/example-program -name Ada -repeat 2

See examples/README.md in the SDK for the reading order, suggested experiments, and service requirements. Start with example-cosmic-syntax for inferred declarations, concise functions, and collection loops. The database lesson covers generated rows and queries; the compute lesson combines CPU and GPU functions in one Cosmic source file. Graphics and compute require macOS, and GPU execution requires an available Metal device.

The HTTP and server lessons supply their own local peers. The WebSocket, messenger, and PostgreSQL client lessons require a suitable service as described in the examples' README. Application options such as -name use a single hyphen.

Minimal program

Save this as main.cpp. It creates a CProgram from the command-line arguments and a null cvar, then exits successfully without printing anything:

#include <mc/CProgram.h>
#include <mc/cvar.h>

int main(int argc, char** argv){
  mc::CProgram program(argc, argv);
  mc::cvar value;
  return 0;
}

Build it with the CMakeLists.txt and commands in SDK setup, then run it with MC_HOME set to the extracted SDK:

MC_HOME="$CATALYST_SDK" ./build/my_app

CProgram defaults cArgs().at("#name") to cBasename(cExecutablePath()), including any filename extension. Configuration lookup uses that actual executable name even when launched through a symbolic link or a substituted argv[0]. Positional argument zero retains the original invocation text. --name chosen selects chosen.ccfg explicitly; constructors taking a cvar argument map also accept "#name". Reconfiguration retains the selected name across executable upgrades.

Errors and stack traces

In both Release and Debug builds, CError remembers the stack where the error was constructed. After your application initializes CProgram, an uncaught CError prints its message and that stack to standard error before the process terminates. This includes errors escaping a worker thread and errors during CProgram initialization.

#include <mc/CProgram.h>
#include <mc/error.h>

void loadSettings(){
  throw mc::CError("settings file is missing");
}

int main(int argc, char** argv){
  mc::CProgram program(argc, argv);
  loadSettings();
}

A Debug build with source information produces output like this (addresses and additional system frames are omitted here):

Uncaught mc::CError: settings file is missing
Stack trace (most recent call first):
#0 loadSettings() at /path/to/main.cpp:5
#1 main at /path/to/main.cpp:10

Copies and rethrows retain the original construction stack. Construct an error where the failure occurs: throwing an error object created earlier reports that earlier location. Catalyst errors you catch are silent unless you explicitly log them. To report a caught error:

#include <iostream>

try{
  loadSettings();
}
catch(const mc::CError& error){
  std::cerr << error.what() << '\n' << error.stackTrace();
}

what() and str() still return only the message. stackTrace() returns a cstr, or an empty string when capture is disabled or a trace cannot be obtained. Diagnostic failures do not replace the original error. To inspect the current call stack instead, use CProgram::stackTrace().

Stack capture does not require -g or debug-symbol bundles. The SDK's optimized Release libraries report available function symbols and addresses, without framework source code or file-and-line information. Optimization can inline or remove calls, and stripped functions may appear only as addresses or offsets from the nearest available symbol.

Build with source information

For your application, configure CMake with -DCMAKE_BUILD_TYPE=Debug and call catalyst_debug_symbols(my_app) after creating the target. Call it for each of your shared-library targets too. On macOS this generates matching .dSYM bundles; keep them beside their binaries when copying or installing your application.

The SDK contains optimized Release libraries without debug symbols. Debugging your application does not add source information to those libraries. Your application's frames can include source file names and line numbers while framework frames continue to show symbols and addresses.

MC_ERROR_STACK_TRACES defaults to 1 regardless of NDEBUG. Keep this setting consistent across the application and its libraries; the SDK uses the enabled default. For custom macOS build systems, generate matching debug bundles for your executables and shared libraries when you want source locations in their traces.

Scope and limitations

Traces retain up to 64 stack addresses; very deep stacks are truncated. Compiler optimization and missing symbols can reduce the available frames or source detail. When Catalyst translates an external exception to CError, its trace starts at that translation; it cannot recover calls already unwound by the external exception.

CProgram installs its reporting handler on first use of its program services, including option registration. It calls the previous terminate handler after reporting, so an existing crash reporter or the C++ runtime may print additional diagnostics. A later call to std::set_terminate() replaces Catalyst's handler. Arbitrary C++ exceptions and fatal signals do not acquire saved CError traces.

Priority queues

CPriorityQueue<Item> is a bounded, lock-free queue for multiple producers and consumers. Submit a value and a double priority with push(item, priority); pop(item) removes the highest-priority value currently queued. Use pop(item, priority) to retrieve its priority too.

Include <mc/queue.h> and link your CMake target with Catalyst::Catalyst.

#include <mc/queue.h>

mc::CPriorityQueue<int> queue(1024);
if(!queue.push(42, 5.5)){
  // Keep the work and retry later, or handle unavailable queue storage.
}

int item;
double priority;
if(queue.pop(item, priority)){
  // item == 42 and priority == 5.5 in this example.
}

Higher numbers come first, including when priorities are negative. Equal priorities retain insertion order; overlapping pushes may be ordered either way. Positive and negative zero have equal priority. Positive and negative infinity are supported. NaN is rejected with CError, without inserting the item.

The queue copies simple, trivially copyable values, including pointers and ordinary scalar types. For a complex object such as cvar, use a pointer, for example CPriorityQueue<cvar*>. A pointer is borrowed: the queue neither deletes the object on pop nor deletes queued objects when the queue is destroyed. Keep pointed-to objects alive until their users finish. Null pointers can be queued, so use the Boolean return from pop to distinguish success from an empty queue.

The constructor allocates all queue storage. The default capacity is 1,024 entries; supply a positive capacity for your workload and inspect it with capacity(). Valid pushes, pops, and empty() allocate no memory. Always check the Boolean result of push: storage may be full or temporarily held by concurrent operations, even after an item has been removed. A failed push leaves ownership with the caller and does not insert anything.

pop returns immediately when no item is available and leaves its output arguments unchanged. Operations may retry under contention; lock-free does not guarantee a fixed completion time for each caller. empty() is a momentary observation, so call pop directly when retrieving work. New higher-priority work can arrive after a pop has selected its item.

Insertion cost grows linearly with queue length and, in the worst case, capacity. An uncontended pop takes constant time. This makes capacity and typical queue depth useful considerations when choosing this queue for a workload. Stop all users before destroying the queue. The queue cannot be copied or moved.

Containers and C++23

Catalyst containers use standard C++23 method names for standard operations, alongside Catalyst conveniences. Include the header named after the type, such as <mc/CVector.h>. Related methods and their overloads are grouped in each header: construction, assignment, iterators, capacity, element access, lookup, modifiers, container operations, and output.

Use ranges

Construct owning sequences and associative containers with std::from_range. Sequences support assign_range, append_range, and insert_range; lists and deques also support prepend_range. Associative containers accept insert_range, and strings also provide replace_with_range. These operations consume the supplied range immediately. Views over an existing container retain their usual lifetime and iterator-invalidation requirements.

#include <array>
#include <ranges>
#include <mc/CVector.h>

std::array source{1, 2, 3, 4};
auto even = source | std::views::filter([](int n){ return n % 2 == 0; });
mc::CVector<int> values(std::from_range, even); // {2, 4}
values.append_range(std::array{6, 8});           // {2, 4, 6, 8}
values.insert_range(values.begin(), std::array{0});
mc::erase_if(values, [](int n){ return n > 4; }); // {0, 2, 4}

Use mc::erase, mc::erase_if, and mc::swap where the corresponding container supports them. Standard algorithms can work with compatible Catalyst iterators. An API that requires a particular standard container type, including qualified helpers such as std::get and std::getline, may require .std() explicitly. CFlatSet::std() returns a live std::flat_set reference. Use stdCopy() for an independent standard flat set; changes to that copy do not affect the original.

Existing Catalyst conventions

Methods such as has, popBack, and dump remain available. For example, popBack() returns the removed value, while standard pop_back() returns nothing. Numeric-index overloads such as CVector::erase(index) coexist with standard iterator overloads. CFlatSet::replace(oldKey, newKey) replaces one key, while replace(container) replaces the complete sorted, unique sequence. CFlatMap::keys() returns a reference; use keysCopy() for an independent copy.

The specialized CSVector, CPVector, and CVectorSet retain their own contracts. In particular, CVectorSet::erase(value) returns nothing, and a duplicate insert(value) returns {end(), false}. Standard sets return an erased count and an iterator to the existing element, respectively. These existing meanings have been preserved. The compatibility reference lists the remaining differences and name overlaps.

Container operations translate escaping failures into CError subclasses, including COutOfRangeError and CLengthError. Operations performed directly on iterators, elements, or a standard container obtained through .std() follow their own exception rules. CArray checks excess initializer-list elements at runtime, rather than rejecting them during compilation as std::array does.

Values and CSON

mc::cvar holds a Catalyst value. mc::cvec is a vector of values, and mc::cmap maps string keys to values. Maps are useful for named fields; vectors preserve order.

#include <mc/cvar.h>
#include <mc/CSONParser.h>

using namespace mc;

cvar user = cmap{{"name", "Ada"}, {"active", true}};
user["name"] = "Ada Lovelace";

CSONParser parser;
cvar settings = parser.parse(R"({theme: "dark" sizes: [12 16 24]})");

CSON accepts quoted strings, numbers, booleans, nulls, maps, vectors, sets, symbols, and expressions. Commas are optional. Bare value names become symbols, so write type: "integer" with quotes in a database description. Parsing an expression preserves it without evaluating it.

parse(code, config = false) accepts either a null-terminated const char* or a cstr. parseFile(path, config = false) reads a file. Each call must contain one complete value or expression. Results own their contents and remain valid after the input or parser is destroyed. You can reuse a parser after either a successful parse or an error.

CSON also accepts single-quoted strings, // and /* ... */ comments, and = between a map key and its value. A map head such as {widget, width: 20} stores "widget" under the key "#". none denotes the missing-value sentinel.

Configuration strings

Pass true as the second argument to expand $(NAME) from the process environment inside string values. This applies to nested maps, vectors, sets, and function arguments. Map keys and symbols remain literal. Expansion does not evaluate expressions or change the types of values.

CSONParser parser;
cvar settings = parser.parse(
  R"({cache: "$(HOME)/Library/Caches/my-app"})", true);
cvar fileSettings = parser.parseFile("settings.cson", true);

Adjacent references and variables with empty values are supported. A variable's replacement text is inserted once; references inside that text are not expanded again. An undefined variable retains its $(NAME) spelling and emits a warning. In a CSON string, write "\\$(NAME)" to suppress expansion; the parsed string retains the backslash. Ordinary parsing leaves all references literal.

Tokens and parsing errors

parser.tokens() returns classified source spans from the most recent parse, including names, numbers, strings, and operators. Each CToken has a type, byte index, and byte size in the original input. Long strings occupy adjacent tokens. Copy the token vector if you need to keep it across another parse; tokens do not retain the input text.

Syntax errors throw CParseError, with a one-based source line and, for files, the path. File access errors throw CError. Inputs are limited to 16,777,215 bytes by the token offset representation. Excessively nested input or value trees report a parse error. Use a separate parser for each concurrent call.

Use value.isMap(), isVec(), and other type checks before accessing an unknown value. Use value.at("name") for an existing map field and value.getVec().at(0) for a vector element. cNone is a distinct missing-value sentinel; a default-constructed cvar is null.

Native values and callable functions

cvar::wrap(value, kind) holds a C++ value inside a cvar. An lvalue is copied and an rvalue is moved. Retrieve it with unwrap<T>(description); a type mismatch throws CError. The description appears in the error message.

cvar name = cvar::wrap(cstr("Ada"), 900);
name.unwrap<cstr>("name") += " Lovelace";

auto resource = std::make_unique<int>(42);
cvar owned = cvar::wrap(std::move(resource), 901);

Copies of a wrapped value share the same object. They do not copy its contents again. The object is released when its final owning cvar is released. Synchronize access to shared mutable contents. The kind argument is application metadata; it does not replace the C++ type check.

Use Func0 through Func15 to store callable functions taking zero through fifteen const cvar& arguments and returning a cvar.

cvar increment = cvar::wrap(
  cvar::Func1([](const cvar& value) -> cvar{
    return value + 1;
  }), cvar::Lambda1);

cvar result = increment(41); // 42
cvar functions = cmap{{"increment", increment}};
result = functions["increment"](9); // 10

The supplied callable must use the matching FuncN type. An incorrect argument count, an empty function, or a callback failure reports CError. Lambda0 through Lambda15 identify these callable kinds. A callable executes when invoked; an ordinary symbolic cfunc remains an unevaluated expression.

Wrapped objects, callables, and iterators are process-local C++ values. They cannot be written with store(), pack(), or save(), including when nested in a collection. Convert their contents to ordinary values before saving or sending them. Python and Swift bindings continue to exchange portable value snapshots and reject these native objects.

Custom objects

Derive from CObject and implement execute(const cfunc&) to give an object behavior through a cvar. The function name selects the operation and its arguments provide the inputs. Check both the name and argument count.

class Counter : public CObject{
public:
  cvar execute(const cfunc& f) override{
    if(f.name == "Add" && f.args.size() == 1){
      return value_ + f[0];
    }
    if(f.name == "AddBy" && f.args.size() == 1){
      value_ += f[0];
      return cNone;
    }
    if(f.name == "Numeric" && f.args.empty()){
      return value_;
    }
    return fail(f);
  }
private:
  cvar value_ = 0;
};

cvar counter(new Counter);
counter += 3;
cvar next = counter + 2; // 5; counter still holds 3
double count = counter.as<double>();

Use value.execute(cfunc("method", arguments...)) to invoke another method directly. Unsupported operations and exceptions from custom code report CError. size(), empty(), and span() can now throw when an object's method fails.

Operations returning a C++ reference, including at(), numeric indexing, front(), back(), and reference conversions, require the object to return a borrowed cvar reference to live storage, for example return &memberValue;. That storage must remain valid for the caller's use. Returning a temporary value is rejected.

Implement canExecute() to advertise optional behavior. A supported Str() supplies the display string. Objects defining equality must also supply Hash() to be used as set elements. Return an ordinary value representing their equality: equal objects must produce equal hash values, and values affecting equality must remain unchanged while stored in a set.

Iterators and erasure

begin_(), end_(), and find(value) return iterators held in cvar values. They support vectors, function arguments, sets, maps, and custom objects. Link your target to Catalyst::Shared to use these operations. The ordinary C++ begin() and end() remain available.

cvar values = cvec{10, 20, 30};
for(cvar it = values.begin_(), end = values.end_(); it != end; ++it){
  cvar value = it.Star();
  // Use value here.
}

values.erase(values.find(20));
values.erase(values.begin_(), values.end_());

cvar text = "abcdef";
text.erase(2, 2); // "abef": byte position and byte count

find() searches sequence values, set members, or map keys. A miss returns the end iterator. Star() returns a borrowed reference to a vector or function argument, a copy of a set member, or a map with key and value snapshots for a map entry. Set keys cannot be mutated through an iterator.

Copying an iterator cvar shares its position; it++ returns an independent iterator at the previous position. These are forward iterators. Incrementing or dereferencing an end iterator created by these methods throws CError.

Keep the container alive and follow its usual iterator invalidation rules. Structural edits, reallocation, rehashing, replacement, and erasure can invalidate iterators; obtain fresh ones afterward. Iterators do not extend the container's lifetime. An erase range is [first, last); both endpoints must belong to that container and occur in forward order. An empty range is allowed.

Python provides value.erase(position, count=1); Swift provides try value.erase(at: position, count: count). These erase vector/function elements or string bytes. Counts are clamped at the end, negative inputs are rejected, and existing snapshots remain unchanged. For UTF-8 text, choose byte boundaries that preserve whole characters.

Construct runtime objects

cvar::construct(name, arguments...) creates a registered object. Give the returned pointer to an owning cvar. Unknown class names and invalid constructor arguments throw CError. Link with the supplied core library to use these factories. The Microcosm interpreter initializes the built-in factories when it first constructs a native object or imports a module; no separate initialization call is required.

cvar queue(cvar::construct("Queue_cms"));
queue << 42;
cvar item = queue.execute(cfunc("pop"));

cvar random(cvar::construct("Random_cms", 123));
cvar sample = random.execute(cfunc("uniform"));

On macOS, link Catalyst::MacShared to construct Draw_cms, Plot_cms, and Movie_cms. Drawing and movie constructors take width and height; plotting takes a plot configuration. Drawing fonts use a name followed by a size, such as Helvetica 18. Drawing render(path) and plotting save(path) write JPEG images; movie render(path, fps) exports video. Plotting setData(name, values) copies its input.

A messenger's receive callback uses Func1 and returns a Boolean indicating whether it handled the message. Its close callback uses Func0. Passing null clears a callback. execute(cfunc("callbackError")) retrieves and clears a callback failure by throwing CError. A receive timeout returns null. A receive-callback failure closes that connection; construct a new messenger to reconnect. Finish calls before releasing their objects, and release captured resources when callbacks no longer need them.

Unix timestamps and local time

Include <mc/time.h> to construct, inspect, parse, and format timestamps on macOS and Linux. The Unix epoch is January 1, 1970 at 00:00:00 UTC. Calendar fields and formatted strings use your process's local time zone. To use UTC throughout a program, start it with TZ=UTC0 ./my_program.

#include <mc/time.h>

double seconds = mc::cEpochMake(2024, 7, 1, 12, 30, 45.25);
uint64_t nanos = mc::cMakeNano(2024, 7, 1, 12, 30, 45, 250000000);

int year, month, day, hour, minute, second;
mc::cEpochUnmakeSeconds(seconds, year, month, day, hour, minute, second);
mc::cEpochUnmakeNano(nanos, year, month, day, hour, minute, second);

mc::cstr text = mc::cNanoTimeStr(nanos); // "2024-07-01 12:30:45"
mc::cstr today = mc::cTimestamp("%Y-%m-%d");
uint64_t midnight = mc::cNanoTime("2024-07-01", "%Y-%m-%d");

Formatting and parsing use the standard put_time and get_time format syntax. The process's time zone determines calendar conversion; a format string does not select a different zone. Parsing does not accept fractional seconds. Use cMakeNano() when supplying a nanosecond fraction.

The original cEpochUnmake() overloads remain available: double arguments mean seconds and integral arguments mean nanoseconds. Integer literals are accepted, and negative integral arguments report CError. Prefer the explicit unit names in new code, especially when passing values between functions.

Invalid dates, invalid clock fields, nonfinite seconds, conversion failures, and values outside the destination's range report CError in every build configuration. Dates are not normalized: February 29 in a non-leap year is an error. Unix nanoseconds must lie between zero and UINT64_MAX, ending at 2554-07-21 23:34:33.709551615 UTC. Earlier or later timestamps are rejected rather than wrapped. Second-based conversions are also limited by the platform's local calendar support.

Daylight saving time is determined automatically. A nonexistent local time, such as 02:30 during a spring transition that skips that hour, is rejected. When an hour repeats in autumn, conversion uses the platform's choice of occurrence. Preserve the Unix timestamp when the particular occurrence matters; plain local calendar text cannot distinguish them. These functions can be called concurrently while the process's time zone and locale remain unchanged.

Dates across billions of years

cMakeTime() and cUnmakeTime() represent dates as exact integer seconds over a range large enough to describe time since the Big Bang. Their fixed epoch is January 1 at 00:00:00 in year CTimeEpochYear, which is −13,700,000,000. That instant has value zero. The epoch is a calendar convention and does not change with scientific estimates of the universe's age.

uint64_t beginning = mc::cMakeTime(mc::CTimeEpochYear); // 0
uint64_t first = mc::cMakeTime(2024, 12, 31);
uint64_t next = mc::cMakeTime(2025, 1, 1);
uint64_t elapsedSeconds = next - first; // 86400

int64_t year;
int month, day, hour, minute, second;
mc::cUnmakeTime(next, year, month, day, hour, minute, second);
// year = 2025, month = 1, day = 1; clock fields are zero.

The timeline applies Gregorian leap-year rules to every year, including dates before the Gregorian calendar was adopted. It includes astronomical year zero (1 BCE); year −1 is 2 BCE. Every day has exactly 86,400 seconds. These functions do not apply time zones, daylight saving time, or leap seconds, and they do not reconstruct historical calendars. The Unix-time helpers likewise do not accept a seconds field of 60.

All uint64_t timestamp values can be decoded and encoded again without losing a second. Dates before the fixed epoch or beyond the timestamp range report CError. Keep these values as integers: converting a present-day cosmic timestamp to double loses second-level precision. The unsigned range spans about 584 billion years in seconds; using nanoseconds would reduce that span to about 584 years. Compare timestamps before subtracting if their order is unknown, because unsigned subtraction cannot express negative durations.

Calendar-only helpers are independent of the local time zone: cGetDayInfo(year, month, day, dayOfYear, dayOfWeek) returns a one-based day of year and Sunday-based weekday. cDayOfYear(year, dayOfYear, month, day) performs the inverse ordinal-date lookup and takes an unsigned year. Both validate their input, including leap days. These helpers and the cosmic conversions can also be used in constant expressions.

Cosmic timestamps and Unix timestamps have different epochs. Do not pass a value from cMakeTime() to a Unix-time helper. Values saved using the earlier, incorrect cosmic conversion should be recomputed from their original dates: that conversion mapped some different dates to the same value, so recovery from a timestamp alone is not always possible.

Connect to a database

#include <mc/CDB.h>

mc::CDB db("dbname=catalyst_demo connect_timeout=5");

CDB accepts PostgreSQL connection strings, including connection URIs. Construction opens the connection and reports failures with a Catalyst exception. For a remote database, provide the host, database, user, and appropriate PostgreSQL authentication/TLS options. PostgreSQL password files and service definitions can keep credentials out of application source.

Each instance represents one connection. Calls on an instance must be serialized. Use separate instances or application-managed connections for concurrent work. Rows copied or returned by CDB own their values and can outlive the connection.

Describe and create tables

createTable() accepts a map-valued cvar, a cstr containing CSON, or a CSON string literal. Each describes one table:

db.createTable(R"({
  schema: "public"
  name: "users"
  columns: [
    {name: "id" type: "integer" nullable: false identity: true}
    {name: "accountId" type: "integer" nullable: false}
    {name: "name" type: "string" nullable: false}
    {name: "active" type: "bool" nullable: false default: true}
    {name: "settings" type: "map" default: {theme: "dark"}}
  ]
  primaryKey: ["id"]
  indices: [
    {name: "users_account_name" columns: ["accountId" "name"] unique: true}
  ]
})");

Each column requires name and type. nullable defaults to true. An omitted default means no explicit default; default: null specifies a null default. Literal defaults use the column's declared type, including serialized collection values.

identity: true is available for integer columns. PostgreSQL generates a value when the field is omitted; explicitly supplied values are allowed. Identity columns cannot also specify a default. Sequence values may have gaps, including after rolled-back inserts.

For a SQL expression default, use defaultSql instead of default, for example defaultSql: "40 + 2". This field is SQL source, so supply an application-controlled expression.

Unknown fields, duplicate columns, unsupported types, and conflicting options produce errors. Existing tables are not overwritten. Creating a table and its indices is atomic, including inside a manual transaction.

Names in helper methods preserve case. A name can be schema-qualified, such as public.users. Quote a component containing a literal dot, for example "public"."sales.2026". Identifiers are limited to 63 UTF-8 bytes. Raw SQL follows PostgreSQL's quoting rules; the mixed-case accountId column needs double quotes there.

Choose column types

Use native columns for fields you filter, sort, join, aggregate, or use as keys. Use cvar when different rows need different value types, when a string contains NUL bytes, or when tag metadata must be preserved. Native columns reject metadata they cannot represent and report mismatched value types as errors.

Serialized values preserve nested collections, symbols, expressions, packed values, binary buffers, and type metadata through Catalyst's store() representation. Buffer contents are saved in full, independently of their current read position. Expressions are not evaluated. Untagged nulls become SQL NULL; cNone remains a separate serialized value. Nullability is enforced by the row helpers even for tagged nulls.

Buffer values must be contiguous bytes. Flatten structured buffers with attached child buffers before storing them.

References are saved as snapshots of their target values. Object values are unsupported because Catalyst does not define their serialization. Stored pointer addresses do not become valid objects in another process. Cyclic values are rejected; nesting is limited to 256 levels.

The contents of a serialized field are opaque to SQL. Comparing or indexing its bytes does not provide Catalyst value equality: maps and sets do not have a canonical byte ordering. Put searchable attributes in separate native columns. PostgreSQL fields are limited to approximately 1 GB, so large external assets should be stored separately.

Insert data

mc::cvar user = mc::cmap{
  {"accountId", 7},
  {"name", "Ada"},
  {"settings", mc::cmap{{"theme", "light"}}}
};

uint64_t inserted = db.insert("users", user);

A row is a map from column names to values. Missing columns use their defaults, generated identities, or SQL null where permitted. An empty map requests an all-default row. An explicit null writes SQL null. Unknown columns and incompatible values are errors. The return value is the number of inserted rows, not a generated identifier.

For bulk loading, pass a vector of row maps. All rows must have the same nonempty set of keys. Empty vectors insert zero rows. A batch succeeds completely or rolls back completely.

mc::cvec rows{
  mc::cvar(mc::cmap{{"accountId", 7}, {"name", "Grace"}}),
  mc::cvar(mc::cmap{{"accountId", 7}, {"name", "Margaret"}})
};
db.insert("users", mc::cvar(std::move(rows)));

Send large imports in bounded batches so the application does not need to build a vector containing the entire dataset.

Select and traverse rows

Use SQL to select the needed columns and filter the needed rows. Values are supplied separately through PostgreSQL's $1, $2, … parameter syntax.

db.queryMap(
  R"(SELECT id, name FROM users
     WHERE "accountId" = $1 AND id > $2
     ORDER BY id LIMIT $3)",
  mc::cvec{7, 0, 1000},
  [](const mc::cmap& row){
    std::cout << row.at("name") << '\n';
    return true;
  });

queryMap() supplies a map keyed by selected column names or aliases. Duplicate names are errors, including for empty results; use aliases when joining tables. query() supplies a vector in selected-column order and allows duplicate names.

• Return true from the callback to continue, or false to stop.
• The query method returns true on exhaustion, including an empty result, or false when the callback stops it.
• Rows are borrowed for the duration of the callback. Copy a row to retain it. Copying keeps nested values and buffers alive.
• Traversal fetches bounded batches of 256 rows. Stopping prevents further fetches, although the current batch may already be received.
• Callbacks run synchronously. Use a different CDB for database work inside a row callback; re-entering the same connection is an error.
• Stopping traversal does not commit or roll back an enclosing transaction. Exceptions stop traversal and propagate.

These methods accept a single cursor-compatible SELECT or VALUES query, including ordinary SELECT CTEs. Data-changing statements and data-changing CTEs belong in execute(). SQL functions can still have their usual side effects.

Direct serialized columns, including aliases and joined table columns, are decoded to their Catalyst values. A SQL expression returning bytea, such as coalesce(settings, settings), produces an ordinary buffer because it no longer identifies a declared stored-value column. If that expression is known to contain serialized data, restore it explicitly:

mc::cvar row = db.get("SELECT coalesce(settings, settings) FROM users LIMIT 1");
if(!row.isNone() && !row.getVec().at(0).isNull()){
  mc::cvar settings(row.getVec().at(0).getBuf());
}

In raw SQL parameters, scalars use native representations, buffers use raw bytes, and collections or tagged values use serialized bytes. Provide a SQL cast when an expression has no inferred parameter type, such as SELECT $1::bigint or SELECT $1::bytea. Use the row helpers to store scalar values in a cvar column; they apply the declared column representation.

Other SQL result types, such as exact decimals and timestamps, return their PostgreSQL text representation. They are not silently rounded into Catalyst numeric values.

Get the first row

mc::cvar user = db.getMap(
  R"(SELECT id, name, settings FROM users
     WHERE "accountId" = $1 ORDER BY id)",
  mc::cvec{7});

if(user.isNone()){
  std::cout << "No matching user\n";
}
else{
  std::cout << user.at("name") << '\n';
}

get() returns a cvar containing a vector; getMap() returns one containing a map. Both return cNone when no row exists. A row whose fields are all null is still a row. The returned value owns its contents.

Only the first row is fetched. Extra rows are allowed; these methods do not check uniqueness. Use ORDER BY with a unique tie-breaker when you need a deterministic first row. A query's sort or aggregation may still require substantial server work before that first row exists.

Update and erase rows

db.update("users",
  mc::cvar(mc::cmap{{"active", false}}),
  mc::cvar(mc::cmap{{"id", 42}}));

db.erase("users", mc::cvar(mc::cmap{{"id", 42}}));

update(table, changes, match) changes the specified fields. Omitted fields remain unchanged. erase(table, match) deletes matching rows. Both return affected-row counts; zero is a successful operation with no matches.

match must be a nonempty map of native-column conditions. Conditions are combined with AND. Ordinary values use equality; null uses IS NULL. Updates require a nonempty changes map. Use parameterized SQL for more expressive conditions:

db.execute(
  "UPDATE users SET active = false WHERE id < $1",
  mc::cvec{100});

db.execute("DELETE FROM users WHERE active = $1", mc::cvec{false});

execute() accepts one SQL command without result columns and returns its affected-row count. Use it for DDL and general INSERT/UPDATE/DELETE commands. Use query()/get() for row results. DML with RETURNING is not supported by these traversal methods. Use CDB's transaction methods for transaction control and insert() for bulk loading.

Commit and roll back

Autocommit is enabled initially. Each successful operation commits independently. Related operations can share a transaction using either explicit control or a transaction callback.

Explicit transaction

db.begin();
try{
  db.insert("users", firstUser);
  db.insert("users", secondUser);
  db.commit();
}
catch(...){
  db.rollback();
  throw;
}

Manual-commit mode

db.setAutoCommit(false);
db.insert("users", firstUser);
db.insert("users", secondUser);
db.commit();

With autocommit disabled, the next database operation starts a transaction automatically. This includes reads and schema inspection. It remains active until commit() or rollback(). After either call, the next operation starts a new transaction.

Transaction callback

db.transaction([&](mc::CDB& tx){
  tx.insert("users", firstUser);
  tx.insert("users", secondUser);
});

Normal return commits; an exception rolls back. Transaction callbacks cannot be nested or take over transaction control with begin(), commit(), rollback(), or setAutoCommit().

• inTransaction() reports an active transaction, including an aborted one. autoCommit() reports the configured mode.
• Changing autocommit mode requires no active transaction.
• commit() and rollback() are no-ops when no transaction is active.
• A server SQL error normally aborts a manual transaction. Further operations and commit report an error until you roll back. Atomic table creation and bulk insertion isolate their failures, preserving preceding work in an enclosing transaction.
• Destroying a CDB rolls back pending work. It never commits pending work as a side effect of destruction.
• A connection failure during commit can leave the outcome uncertain. CDB reports the failure and does not retry writes automatically.

Transactions use PostgreSQL's Read Committed isolation by default. A single query has a consistent query snapshot; successive statements can observe newly committed changes. Keep transactions short enough to avoid holding unnecessary locks and old row versions.

Create and remove indices

mc::CSONParser parser;
db.createIndex("users", parser.parse(R"({
  name: "users_active_id"
  columns: ["active" "id"]
})"));

db.dropIndex("users_active_id");

An index description requires name and an ordered, nonempty columns vector. unique defaults to false. method defaults to btree; hash, brin, gin, gist, and spgist can be requested where PostgreSQL supports them for the chosen columns.

For a populated table, concurrently: true requests an index build that allows ongoing writes. This requires autocommit and no active transaction, and cannot appear in a table's initial index list. A failed concurrent build may leave an invalid index; inspect it and drop it before retrying. Partial indices, expression indices, descending keys, and other advanced options can be created with explicit SQL.

A primary key supplies its own index. Additional indices cost storage and work on inserts and updates; choose them for actual query patterns.

Read table schemas

mc::cvar tables = db.schema();
mc::cvar users = db.schema("public.users");

// Recreate a supported definition in another database.
if(!users.isNone() && !users.getMap().has("unsupported")){
  destination.createTable(users);
}

schema() returns a vector of table descriptions, ordered by namespace and table name. It includes visible user tables across SQL namespaces and excludes PostgreSQL system tables. schema(table) returns one description, or cNone when the table is absent.

The descriptions use the same structure as createTable(). Column order, nullability, identity, supported defaults, primary keys, and indices are included. Defaults are normalized; formatting and CSON comments are not retained. Index descriptions are ordered by name and omit the implicit primary-key index. Construction options such as concurrently are not part of a stored schema.

Schema reads reflect current SQL definitions, including renames and changes made with SQL. A changed or externally defined default may be reported as defaultSql. A description with unsupported contains a list of features outside the current recreation vocabulary, such as foreign keys, custom SQL types, generated columns, partitioning, triggers, or advanced index options. CDB refuses to create a table from such a description. Use PostgreSQL schema tools when a complete migration of those features is needed.

Remove a table

db.dropTable("users");
db.dropTable("old_users", true);  // Missing table is acceptable.

This removes the table, its rows, and its own indices. The default is to report a missing table as an error. Pass true as the second argument to tolerate absence; dropIndex() offers the same option.

Dependencies from other objects prevent removal. If cascading removal is intended, issue explicit SQL. Normal table and index removal follows the current transaction mode and can be rolled back before commit.

Errors and connection lifetime

try{
  db.insert("users", user);
}
catch(const mc::CError& error){
  std::cerr << error.what() << '\n';
}

Validation errors, SQL errors, connection failures, and callback exceptions surface as CError or a subclass. SQL error messages include PostgreSQL's SQLSTATE. Translated external errors retain their original exception as a cause.

CDB does not reconnect or replay operations automatically. After a broken connection, construct a new instance and resolve any uncertain write outcome at the application level. After an ordinary SQL error in a manual transaction, roll back before continuing.

For a long traversal, some callbacks may already have run when a later fetch fails. A database rollback cannot undo unrelated work performed by those callbacks, such as writing a file or sending a message.

Work with large datasets

CDB can traverse results without accumulating the complete result set in application memory. A billion-row database still needs a schema, storage system, and maintenance plan matched to its workload.

• Select only needed columns and filter in SQL.
• Use indexed keyset pagination, such as WHERE id > $1 ORDER BY id LIMIT $2, for resumable work.
• Use bulk insertion in bounded batches for imports.
• Keep searchable fields in native columns and avoid loading large serialized fields unnecessarily.
• Choose partitioning from access and retention patterns, using PostgreSQL DDL when appropriate.
• Monitor query plans, vacuuming, statistics, disk space, and update throughput.
• Measure representative data sizes and concurrency before relying on a latency or throughput target.

A short client traversal does not bound server work for every query: sorts, joins, and aggregates can require substantial processing before rows are returned.

CDatabase tables and commits

mc::CDatabase stores indexed rows in a local directory. Construct it with a path and true to create a database, or false to open an existing one. Typed tables use CRow32 or CRow64; schema-defined tables return rows as cvec, with the row ID in element zero. Use the same row definition or schema when reopening a table.

Table insert, update, and erase stage changes until commit(). rollback() discards the pending batch. An update requires a live, committed row ID and gives the replacement a new ID. Read it again through an index or traversal before updating it in a later batch.

For repeated changes to one row in a batch, the last change wins. Updating a row does not increase the live row count. Erasing a missing or previously erased row has no effect. Unique indices reject duplicate live values, including duplicates within the pending batch; an erase can free a unique value for reuse in the same commit.

Queries and row-ID collections exclude erased and replaced rows. Traversal callbacks return true to continue and false to stop. Selected fields retain their original positions, with unselected fields returned as null. Collect changes during traversal and commit them after the callback returns. Numeric indices reject NaN values and NaN range endpoints.

Composite indices order by their first field within the group identified by the remaining key fields. Native values and equivalent cvar keys now use the same database hashing rules, including string literals and numeric keys of different widths. This changes composite index hashes from older releases; recreate existing databases when upgrading. Row definitions and Cosmic query syntax are unchanged.

commit() makes a table's pending batch visible in memory. To save periodically, call setSaveInterval(seconds) with a positive interval no greater than one billion seconds. Zero disables periodic saving; this is the default. Changing the interval replaces the previous schedule. Background save errors go to std::cerr, or the stream supplied to setErrorStream(stream).

After releasing table handles, call shutdown_() to finish saving and close the database. It reports save failures to the caller; after correcting the storage problem, call it again to retry. Repeated successful shutdown calls are harmless. Destruction also attempts to save, but reports failures through the error stream. Keep a supplied error stream alive until the database has been destroyed.

Only one database object or process may open a directory at a time. It may have multiple table handles. setMemoryLimit(megabytes) accepts a positive memory budget; active pages and pending batches can temporarily exceed it. Saved files are replaced after successful writes. A save across multiple files is not an atomic snapshot, and crash recovery for an interrupted database-wide save is not provided.

Keep the database alive while using its tables. Finish using table handles before erasing or renaming their table, or shutting down the database. Database and schema-table objects must not be copied.

Some original operations, including get and insert, still declare noexcept. Failures during those calls terminate the process. These declarations remain for interface compatibility.

Method reference

Further PostgreSQL documentation: connections, SELECT, indices, partitioning, and vacuuming.

SQL over local tables

<mc/CSQL.h> provides SQL access to a local database directory. Create one with true, or reopen it with false. A borrowed CSQLDelegate receives each selected row during execute() and must outlive the database.

struct Results : mc::CSQLDelegate{
  void handle(mc::cvec& row) override{
    std::cout << row << '\n';
  }
} results;

mc::CSQL db(&results, "items.db", true);
db.execute("CREATE TABLE items (id INTEGER, name TEXT)");
db.execute("INSERT INTO items VALUES (?, ?)", mc::cvec{1, "first"});
db.execute("SELECT name FROM items WHERE id = ?", mc::cvec{1});

Use ? placeholders with a cvec of values. insert(sql, rows) accepts a vector of parameter vectors. insertCSV() and insertCSVFile() accept the same insert statement and a flag indicating whether to skip a header row. Mutating statements commit their changes; close the database normally to save it.

Numeric strings must contain a complete number. Values outside the declared integer width or the finite range of REAL are rejected before modifying rows. Table creation and deletion record enough information to recover their schema metadata after a failed save. Once the storage problem is corrected, the next operation or reopening reconciles the schema with the completed table operation. Check whether the table exists before retrying a failed change. CSQL objects cannot be copied.

By default results are vectors in SELECT-column order. After setMapped(true), implement handle(mc::cmap&) to receive rows keyed by column name. Copy a row to retain it beyond the callback. Boolean columns are represented as integers 0 and 1. CSQL uses the local SQL implementation; PostgreSQL connections and their transaction API use CDB.

Entries, attributes, and relations

<mc/CES.h> stores entries identified by nonzero IDs, named attributes, and directed relations. Reusing an attribute or relationship name returns its existing ID.

mc::CES store("entries.db", true);
auto earth = store.addEntry();
auto moon = store.addEntry();
auto mass = store.addAttribute("mass");
auto contains = store.addRelationship("contains");
store.setValue(mass, earth, 5.972e24);
store.addRelation(contains, earth, moon);
mc::cvar attributes = store.getEntryAttributes(earth);
auto moons = store.getRelationsFrom(contains, earth);
store.shutdown();

Values can be int64_t, double, or an arbitrary cvar. Use the corresponding typed getter and deletion method. Numeric attributes support equality and inclusive range queries. To replace a value, delete it and set the new value. An omitted relation weight is treated as zero.

deleteEntry() also removes its attribute values and incoming and outgoing relations. Deleting a named attribute or relationship removes the associated values or relations. Call compact() occasionally after substantial deletion. IDs are reserved on disk before being returned or used in new rows; an unsuccessful insertion can therefore leave a gap in the IDs. shutdown() saves and closes the database and may be called repeatedly. It reports save failures and can be retried after the storage problem is corrected. Other operations throw once shutdown has begun. Destruction reports save failures without throwing. If compaction fails, close and reopen the store before using it again. Reopen with CES(path, false); store objects cannot be copied.

Store frames and export movies

mc::CTheater stores movies as sequences of JPEG frames in a local directory. Each movie and each frame can carry cvar metadata. You can retrieve individual frames, reopen the collection later, or export a movie as H.264 video.

Include <mc/CTheater.h> and link Catalyst::Shared. Frame storage and retrieval need no external program. Video export requires ffmpeg with the libx264 encoder available to the command shell. On macOS, install it with brew install ffmpeg. FFmpeg chooses the output container from the filename; use .mp4 for an MP4 movie. See the FFmpeg documentation for its supported containers and encoding options.

Create a collection and add frames

#include <mc/CTheater.h>

int main(){
  mc::CTheater theater("frames", true);
  auto movie = theater.createMovie(
    640, 480, 85, mc::cmap{{"name", "Example"}});

  mc::CVector<unsigned char> pixels(640 * 480 * 4, 255);
  for(int frame = 0; frame < 24; ++frame){
    theater.addFrame(movie, pixels.data(), mc::cmap{{"frame", frame}});
  }

  theater.encode(movie, "movie.mp4", 24, 4000);
}

The constructor's create argument is true for a new directory and false to reopen an existing collection. The new directory must not already exist, and its parent must exist. An empty path is invalid. The class is noncopyable; only one open collection may own a given directory at a time.

createMovie(width, height, quality, metadata) returns a nonzero 32-bit movie ID. Dimensions must be positive and supported by JPEG. Quality ranges from 0 to 100 and defaults to 50; it controls JPEG frame storage quality. Metadata defaults to cnull. Movie information includes #id, #size (a vector containing width and height), and #quality; these reserved entries override values supplied in your metadata. movieInfo(id) returns that information, while getMovies() returns a vector of the information for all movies. Deleting a movie does not make its ID available for reuse.

addFrame(id, pixels, metadata) appends a frame, starting at frame index zero. On macOS and other little-endian systems, pixels are packed RGBA bytes in row order, with no row padding. Big-endian systems use ARGB. The input must contain at least width * height * 4 bytes; a raw pointer does not provide enough information to check its allocation size. The function finishes reading the buffer before returning, so you can then reuse it. JPEG compression is lossy and discards alpha; decoded frames are opaque. Frame metadata defaults to cnone and is stored independently from the movie metadata.

Read frames and maintain a collection

#include <cstdlib>
#include <memory>
#include <mc/CTheater.h>

int main(){
  mc::CTheater theater("frames", false);
  mc::cvar metadata;
  std::unique_ptr<unsigned char, decltype(&std::free)> pixels(
    theater.getFrame(1, 0, metadata), std::free);
  if(pixels){
    // Use the decoded pixels and this frame's metadata.
  }
}

The allocating getFrame(id, index, metadata) overload returns a buffer that you must release with free(). The overload getFrame(id, index, buffer, metadata) writes into your own buffer and returns true on success. Your buffer must have room for the movie's full decoded image. Both overloads restore the frame metadata. A missing movie or frame returns nullptr or false and leaves metadata and caller-supplied pixels unchanged. Invalid or corrupt stored images throw CError.

deleteMovie(id) removes the movie and its frames. Deleting an absent movie is a no-op. Call compact() after substantial deletion to reclaim database space. Changes follow CDatabase's commit and persistence behavior; allow the collection to finish destruction when closing it normally.

Export video and use multiple threads

encode(id, path, fps, bitrate) blocks until export finishes. Frame rate is a positive integer. Bitrate is in kilobits per second and defaults to 100000. H.264 export uses yuv420p, requires even width and height, and requires at least one frame. Export preserves the stored frames, so you can append more or encode again. It refuses to overwrite an existing output file. Invalid settings, unavailable encoders, and failed exports throw CError; a failed export can leave a partial output file.

The optional third constructor argument limits simultaneous encoders for that collection; it defaults to three and must be positive. Additional encoding calls wait for a slot. Frame insertion, retrieval, deletion, and compaction coordinate access across threads. Concurrent appends receive distinct consecutive indices, although their order depends on which call proceeds first. Export includes the frames present when its encoding slot starts; later appends are left for the next export. Deleting that movie while it is being encoded can fail the export. Keep the collection alive until every call using it has returned.

HTTP requests

<mc/CHTTP.h> provides synchronous requests with run(), or background requests with start() followed by await().

mc::CHTTP request(mc::CHTTP::JSON);
request.setURL("https://example.com/data.json");
request.setTimeout(10.0);
request.start();
mc::cvar response = request.await();
int status = request.status();

Select Text for a string, JSON for parsed values, or Buffer for binary data. Returned strings and buffers preserve embedded zero bytes and remain valid after the request object is reused or destroyed. Transport errors and invalid JSON throw; background errors are reported by await(). Check status() for HTTP error status codes.

setPost() accepts text or a cvar encoded as JSON. Use addHeader("Content-Type: application/json") when sending JSON. addPost() and postFile() build multipart forms. setType() selects a request method such as PUT; escape() URL-encodes a value. Configuration changes and escape() throw between start() and await(), even if the transfer has already finished. After await returns or reports a transfer error, the request can be configured and reused. status() may be polled while a request runs; it returns zero until a response code is available. Serialize calls to the other methods. Request objects cannot be copied.

Serve files with CHTTPServer

<mc/CHTTPServer.h> provides an HTTP server for a directory of files, with optional directory listings and executable pages. Link with Catalyst::Shared. The configuration is a cvar map, which can also be loaded from CSON. Supply an existing root directory and a port from 1 to 65535.

#include <iostream>
#include <mc/CProgram.h>
#include <mc/CHTTPServer.h>

int main(int argc, char** argv){
  mc::CProgram program(argc, argv);
  mc::cvar options = mc::cmap{
    {"root", "web"},
    {"port", 8500},
    {"host", "127.0.0.1"},
    {"allowHidden", false},
    {"runExecutables", false}
  };
  mc::CHTTPServer server(options);
  server.start();
  std::cout << "Serving http://127.0.0.1:8500/; press Enter to stop.\n";
  std::cin.get();
  server.shutdown();
}

Create web/index.html before running this example. A relative root is resolved against the working directory at construction. start() starts background request handling and returns; keep the application running while it serves requests. Invalid configuration and startup failures, including a port already in use, throw CError. Configuration is copied at construction; changing the original map does not reconfigure the server.

start() and shutdown() may be called repeatedly, including from different threads while the object remains alive. Calling start() on a running server does nothing; calling it after shutdown starts listening again. Shutdown stops accepting connections and releases the listening port. Requests already accepted may continue until completion or their configured timeouts; shutdown does not wait for them. Destruction also shuts down the listener. The server cannot be copied or moved.

Configuration

Option names are case-sensitive. Boolean options require boolean values, sizes and counts require integers, and timeouts accept integer or floating-point seconds. Timeouts must be finite and between 0.1 and 3600 seconds. Strings must not contain embedded zero bytes.

Requests and files

The server accepts HTTP/1.0 and HTTP/1.1 requests. GET returns file contents, HEAD returns the corresponding headers without a body, and OPTIONS returns the supported methods. Other methods receive 405. Each connection serves one request and then closes. File contents are binary-safe; the filename extension determines the content type. Directory requests without a trailing slash redirect to the slash form while preserving the query string.

URL paths are percent-decoded before lookup. Parent traversal, backslashes, malformed escapes, embedded zero bytes, and disallowed hidden or symbolic-link paths are rejected. Directory listings escape filenames for HTML and URLs and omit inaccessible symbolic links. HTTP/1.1 requests require a Host header; malformed headers are rejected. The current server does not provide TLS, authentication, custom request callbacks, request-body processing, chunked request decoding, range responses, or conditional cache validation.

Executable pages

With runExecutables: true, an executable regular file without an extension is run when requested, including when selected as a directory index. Other regular files are served as stored bytes. Each execution receives no command-line arguments. Its working directory is the directory containing the program, standard input is empty, and standard error is discarded. A successful exit returns standard output as an HTML response with caching disabled. Output is the response body; do not print CGI headers. A HEAD request also runs the program to determine its response headers but omits the body.

The program receives REQUEST_METHOD, REQUEST_URI, SCRIPT_NAME, SCRIPT_FILENAME, QUERY_STRING, DOCUMENT_ROOT, SERVER_PROTOCOL, SERVER_SOFTWARE, and SERVER_PORT environment variables. HTTP_HOST, HTTP_USER_AGENT, and HTTP_ACCEPT are supplied when those request headers exist. The query string retains its URL encoding; the program is responsible for interpreting it. The environment uses PATH=/usr/bin:/bin:/usr/sbin:/sbin and does not copy the application's environment.

Timeouts return 504. A failed execution, nonzero exit, or excessive output returns 500. Timed-out programs and their process groups are terminated. Programs run with the application's permissions; use trusted programs and keep their files and directories under trusted control while the server runs. The file-serving root does not restrict what a program itself can access.

Exchange messages with CWebSocket

<mc/CWebSocket.h> provides a secure WebSocket client with cvar messages. Link with Catalyst::Shared. Supply a hostname, an absolute request path, and an optional port to connect(). The default port is 443. Pass the hostname alone, without wss:// or a port suffix; query parameters belong in the path.

#include <iostream>
#include <mc/CWebSocket.h>

mc::CWebSocket socket;
if(!socket.connect("example.com", "/events")){
  throw mc::CError("could not connect to the WebSocket service");
}
socket.send(mc::cmap{{"subscribe", "updates"}});

mc::cvar message;
if(socket.receive(message, 5.0)){
  std::cout << message << '\n';
}

Replace the hostname and path with those of your WebSocket service. Connections always use TLS, including when an explicit port is supplied; plaintext ws:// connections are not supported. CHTTPServer serves HTTP files and pages and does not itself provide a WebSocket endpoint.

Connection and certificate checks

connect() returns true after the TCP connection, TLS handshake, and WebSocket upgrade succeed. It returns false for an invalid endpoint, a failed connection or handshake, or an attempt to connect an already connected object. A failed attempt can be retried. After a peer disconnects, the same object can connect again; starting a new connection discards unread messages and errors from the previous connection. There is no automatic reconnection.

The client requires TLS 1.2 or newer, verifies the certificate chain and hostname, and supplies the hostname through TLS SNI. It uses OpenSSL's default trust paths. For a private service, install its CA in that trust store, or set SSL_CERT_FILE to a PEM file containing the trusted CA before connecting. The certificate still needs to match the hostname or IP address passed to connect(). OpenSSL's trust store is separate from the macOS Keychain.

SSL_CERT_FILE=/path/to/private-ca.pem ./my_program

Connection establishment and each active write have five-second deadlines. Destruction attempts a normal WebSocket close, allowing one second for the exchange before closing the transport. Resolver cleanup and application callbacks must still finish. The object cannot be copied or moved. Destroy it from outside its delegate callbacks, after other application threads have finished using it.

Sending and receiving values

send(value) writes one JSON text message. Ordinary JSON values—null, booleans, numbers, strings, vectors, and maps—are the simplest interchange format. Strings must contain valid UTF-8; embedded zero characters are encoded as JSON escapes. Symbols, functions, and non-finite numbers use the representations provided by CJSONGenerator, which require corresponding interpretation by the receiving application. Sets, buffers, packed values, pointers, and objects cannot be sent directly; convert them to suitable JSON values first. Excessively nested values are rejected.

Incoming payloads are parsed as CSON, so JSON is accepted alongside CSON syntax. WebSocket fragments are combined before parsing, and ping/pong control frames are handled automatically. A received message may contain at most 16 MiB. Protocol violations or oversized messages close the connection. Payload parsing failures and exceptions from a delegate are reported by receive() as CError; they do not terminate the process.

Queued messages remain available after disconnection until read or discarded by a new connect() attempt. A false receive result leaves the supplied value unchanged. Sending and receiving may run concurrently; if multiple threads receive, each queued message is delivered to one of them.

Handle messages with a delegate

Install a CWebSocketDelegate when some messages should be handled as they arrive. Its handle(socket, message) method returns true to consume that message, or false to place it in the receive queue. Changes made through the mutable message reference are retained when it is queued.

class Updates : public mc::CWebSocketDelegate{
public:
  bool handle(mc::CWebSocket* socket, mc::cvar& message) override{
    if(message.isMap() && message.has("heartbeat")){
      socket->send(mc::cmap{{"heartbeat", "acknowledged"}});
      return true;
    }
    return false;
  }
};

Updates updates;
mc::CWebSocket socket;
socket.setDelegate(&updates);

The delegate is borrowed. Keep it alive until it has been replaced or the socket has been destroyed. setDelegate(nullptr) restores the default behavior of queueing all messages, and delegate() returns the currently installed delegate. Replacing it from another thread waits for an active callback to finish. Callbacks for one socket run sequentially on a background thread. Keep them short; they must not block waiting for the socket's own incoming messages. A blocking receive() inside a callback throws CError; a zero-timeout poll is allowed.

Native messaging and server delegates

<mc/CMessenger.h> and <mc/CServer.h> provide asynchronous messaging and server connections. Link with Catalyst::Shared. Start a CPool for I/O, then pass that pool to your server and messengers.

Implement CServerDelegate::admit(server, messenger, auth) to inspect a connection's first message. Returning false rejects and deletes the pending messenger; returning true transfers ownership to your application and queues {"#":"authenticated"} for the client. Install a CMessengerDelegate and initialize the messenger's session during admission. Keep accepted messengers alive until admission has finished; do not delete one from admit().

CMessengerDelegate::handle(messenger, message) runs when a message arrives. Return true to consume it, or false to leave it for receive(). Native messaging preserves serializable cvar data, including symbols and expressions. Use send() to queue a reply; didClose() reports peer closure or errors, while an explicit local close() is silent.

The server borrows its delegate and pool. Admission callbacks may run concurrently; protect shared state. Destroy the server before its delegate so pending connections and active admission callbacks finish. The application still owns accepted messengers: destroy them while their delegates remain alive, then stop the pool. The runnable Cosmic example in examples/28-server/main.cc demonstrates this order, rejected and accepted admissions, echo messages, and closure. listen(port) binds all IPv4 interfaces; the example's client connects over loopback.

JPEG and image buffers

Include <mc/image.h> and link Catalyst::Shared. cCreateJPEG(pixels, width, height, quality, bytes) encodes four-byte pixels and returns the encoded size through bytes. Quality ranges from 0 to 100. The native pixel order is RGBA on little-endian systems and ARGB on big-endian systems; cCreateJPEG_BGRA() explicitly accepts BGRA instead.

cReadJPEG(jpeg, bytes, width, height) decodes into the native pixel order and reports the dimensions. Release decoded storage with delete[], for example through std::unique_ptr<unsigned char[]>. Release encoded storage from either JPEG encoder with std::free. JPEG is lossy and does not preserve alpha.

cAddImage(dst, src, dstWidth, dstHeight, x, y, width, height) copies a rectangle into a destination buffer. It copies pixels without alpha blending and rejects placement outside the destination. cAddJPEG(dst, jpeg, dstWidth, dstHeight, bytes, x, y) decodes and places a JPEG. Supply sufficient storage for each image's dimensions times four bytes per pixel.

Read CSV data

#include <mc/data.h>

mc::CVector<mc::cstr> columns;
mc::cvec rows = mc::cFromCSV("name,count\nEarth,1\nMoon,2", columns);

The first row supplies column names. Each following row becomes a vector; complete numeric fields become numbers, and other fields remain strings. Quoted commas, doubled quotes, multiline fields, CRLF, and a final row without a newline are accepted. Rows with the wrong number of fields and malformed quoting throw a parsing error.

Link the Mac graphics library

The macOS SDK includes the graphics library libcatalyst_mac. On macOS, mc-cosmic automatically links it together with the core libcatalyst library when producing an executable. With CATALYST_SDK set to the extracted SDK directory as shown in SDK setup:

"$CATALYST_SDK/bin/mc-cosmic" main.cc -o my_graphics_app

Use --link FILE to add other object files or libraries; repeat it for each additional input. Object-only compilation with -c does not link libraries.

In CMake, link with Catalyst::MacShared to use it together with the core library, or select Catalyst::MacFramework for framework integration. Your application uses the Catalyst headers; the CMake targets provide the bundled dependencies.

find_package(Catalyst CONFIG REQUIRED)
add_executable(my_graphics_app main.cpp)
target_link_libraries(my_graphics_app PRIVATE Catalyst::MacShared)

For Cosmic .cc sources, call catalyst_cosmic_directory("${CMAKE_CURRENT_SOURCE_DIR}") after defining your targets. The drawing and rendering classes require an available Metal device. Keep these resource-owning objects in one place; do not copy them by value.

Draw and render images

#include <mc/CDraw.h>

mc::CDraw canvas(640, 480);
canvas.setFillColor(mc::double4{1.0, 1.0, 1.0, 1.0});
canvas.clear();
canvas.setFillColor(mc::double4{0.2, 0.4, 0.9, 1.0});
canvas.fillRect(mc::double2{20.0, 20.0}, mc::double2{100.0, 60.0});
canvas.render();
unsigned char* rgba = canvas.buffer();

CDraw supports paths, shapes, text, clipping, transforms, hit testing, and images. Drawing is completed by render(). buffer() returns width × height × 4 RGBA bytes. You may supply a buffer to the constructor or resize(); keep it large enough and alive until the canvas stops using it. An internally allocated buffer remains owned by the canvas and may change on resize.

Shapes, text, and images are composited in the order submitted. drawImage(data, srcTopLeft, srcSize, dstTopLeft, dstSize) reads tightly packed RGBA bytes. srcSize is the full source image width and height; srcTopLeft selects the first source pixel, cropping through the bottom-right edge. Use {0, 0} to draw the whole image. Source dimensions and positions must be whole pixels within the image. The selected pixels are scaled to the destination size and copied during the call, so the source buffer can be modified or released before rendering. Reusing a source buffer submits its new pixels.

Canvas dimensions must be positive and within the device's Metal texture limits. If resizing throws, the old buffer, dimensions, and pending drawing remain available. A successful resize discards pending drawing and clears a newly allocated buffer. Passing the canvas's own buffer back to resize retains its ownership and requires dimensions that fit the existing image.

<mc/CRender.h> provides 3D rendering. Set a camera and projection, create box or sphere models or load a GLB file, then call clear(), draw(), and render(). Read the RGBA result through buffer() and release models with unload(). The library carries its drawing shaders; no source-tree shader path is required at runtime.

Plot data

<mc/CPlot.h> takes a CSON-style description of dimensions, data columns, and layers. Use symbols for column mappings; strings represent literal values.

using namespace mc;
cvar config = cmap{
  {"width", 640}, {"height", 480},
  {"data", cmap{{"x", cvec{0, 1, 2}}, {"y", cvec{0, 2, 1}}}},
  {"layers", cvec{cmap{{"geom", "line"},
    {"aes", cmap{{"x", csym("x")}, {"y", csym("y")}}}}}}
};
CPlot plot(config);
plot.plot();
unsigned char* rgba = plot.buffer();
plot.setData("y", CPlot::Floats{2.0, 0.0, 1.0});
plot.plot();

The buffer contains the configured width × height × 4 RGBA bytes. It is owned by the plot, and is unavailable before the first plot(). setData() invalidates the previous image; call plot() again and obtain a fresh buffer pointer.

Run Metal compute functions

<mc/CGL.h> provides CGLRun for a compiled .metallib and a named kernel function.

mc::CGLRun run("kernels.metallib", "scale");
mc::CVector<float> values{1.0f, 2.0f, 3.0f};
run << values;
run.setGrid(values.size());
run.run();

Arguments occupy Metal buffer slots in the order added. The kernel must match their element types and layout. setGrid() specifies thread counts, and setThreadGroup() optionally selects the thread-group dimensions. Invalid or overflowing dimensions throw before dispatch.

Use start() and await() for asynchronous work. Input bytes are read at start and results are written back at await. Keep argument storage alive and at the same address through await; do not resize a bound vector. Repeated runs read the current input values. clear() removes argument bindings between runs.

After user arguments, the runtime supplies scratch storage, its total byte count, log storage, its total byte count, and the total thread count. These count buffers hold 64-bit unsigned values. setMemory() and setLog() specify bytes per thread. After await, outputLog(thread) writes that thread's bounded log text. GPU command failures throw from await.

Files, paths, and system helpers

Include <mc/system.h> and link Catalyst::Shared for these helpers on macOS and Linux. Existing inline helpers remain usable with Catalyst::Catalyst. All existing function names are retained.

#include <mc/system.h>

auto path = mc::cJoinPath(mc::cUserHome(), "logs", "app.log");
mc::cCreateDirs(mc::cParentDir(path));
mc::cAppendToFile("started\n", path);

auto temporary = mc::cCreateTempDir();
auto settings = mc::cJoinPath(temporary.path(), "settings.txt");
mc::cSaveAtomic("enabled=true\n", settings);
for(const auto& line : mc::cReadLines(settings)){
  std::cout << line << '\n';
}

Lexical normalization can change the meaning of paths containing a symlink followed by ... Use cCanonicalPath when resolution through the actual filesystem is required.

cFileInfo(path, followSymlinks = false) returns a CFileInfo with type, size in bytes, writeTime in Unix seconds, and POSIX permissions. Types are CFileType::Missing, File, Directory, Symlink, and Other. A dangling link remains a Symlink unless following is requested. cIsSymlink(path), cReadLink(path), and cCreateSymlink(target, path) inspect or create links; relative targets are interpreted from the link's directory.

mc::CWalkOptions options{.recursive = true, .hidden = false,
                         .followSymlinks = false};
auto sources = mc::cFindFiles("src", "*.cc", options);
mc::cWalkDir("src", [](const mc::cstr& path, const mc::CFileInfo& info){
  std::cout << path << '\n';
  return true; // false stops the entire walk
}, options);

cFindFiles matches basenames using *, ?, and [...], returning sorted regular-file paths prefixed by the supplied root. By default both functions recurse, skip names beginning with a dot, and do not follow symlinks. Following includes links to regular files and traverses each physical directory at most once, preventing cycles. Walk callbacks receive link metadata and all entry kinds; they do not receive the root itself. Walk order is unspecified. Permission, traversal, and callback failures propagate as CError; entries are not silently skipped on errors.

cUserHome() uses an absolute HOME value or the OS user record. cHome() continues to mean the framework's MC_HOME. cTempDir() uses the OS temporary directory, and cExecutablePath() locates the running executable. cConfigDir(application = ""), cCacheDir(application = ""), and cDataDir(application = "") return locations without creating them. On macOS, config and data use ~/Library/Application Support; cache uses ~/Library/Caches. Linux uses absolute XDG_CONFIG_HOME, XDG_CACHE_HOME, and XDG_DATA_HOME, falling back to ~/.config, ~/.cache, and ~/.local/share. Application names must be single path components.

cCreateTempFile(dir = "", prefix = "catalyst-") and cCreateTempDir(dir = "", prefix = "catalyst-") reserve unique resources with 0600 and 0700 permissions. Empty dir selects cTempDir(). Their move-only CTempFile and CTempDir owners expose an absolute path() and clean up at scope exit, including a temporary directory's contents. remove() performs checked cleanup; release() returns the path and transfers cleanup to you. Destructors make a best-effort cleanup without throwing. The old cTempPath still generates a name without reserving it.

cHasEnv(name) distinguishes an empty variable from a missing one. cEnv(name, fallback) uses its fallback only when the variable is missing; cUnsetEnv(name) removes it. cExpandEnvs(text, strict = true) expands $NAME, ${NAME}, and $(NAME) once, returning a new string. A backslash escapes a dollar sign. Missing variables throw; strict=false leaves their expressions intact. The existing cReplaceEnvs retains its in-place behavior and syntax. Coordinate environment changes with other threads using the environment.

cMonotonicNow() returns steady-clock seconds; cElapsed(start) measures elapsed seconds from such a value. These are suitable for measuring work, not dates. cNow and cTicks retain their existing behavior.

cHostName() returns the full OS hostname, retaining punctuation and domain components. The old cHost retains its abbreviated form. cDiskSpace(path = ".") returns CDiskSpace{capacity, free, available} in bytes, with available reflecting space available to an unprivileged user. cProcessMemory() returns CProcessMemory{resident, virtualBytes} for this process. cAvailableMemory() estimates available RAM in bytes: free plus inactive pages on macOS, MemAvailable on Linux. Memory and disk snapshots may change immediately after the call.

cFileToStr and cSaveBinary now check complete I/O. CSBuffer checks mapping failures and closes the file descriptor after mapping. Its copies still alias the same mapping; destruction does not release it. Call its new close() exactly once per mapping, after every alias is finished, and before reusing an owning instance for another mapping.

Run commands and capture output

CCommand runs shell commands on macOS and Linux. It is header-only and is available through <mc/CCommand.h>. Link your CMake target with Catalyst::Catalyst.

#include <mc/CCommand.h>

mc::CCommand command("printf 'Hello from Catalyst\\n'",
                     mc::CCommand::Output | mc::CCommand::Error);
int result = command.await();
mc::cstr output, error;
command.readOutput(output);
command.readError(error);

The constructor starts the command immediately. By default it uses /bin/bash -lc, which reads login-shell startup files. Add Zsh to use /bin/zsh -ic, which reads interactive-shell startup files. Install the selected shell at that path; zsh is optional and may need installing on Linux. Startup files can affect the command's environment and captured output. Commands use shell syntax, including quoting, pipes, expansion, and redirection.

Zsh terminal job control is disabled for commands with redirected or nonterminal input. Interactive startup files still load in this mode.

For a direct argv launch, use CCommand::fromArgs({"git", "status", "--short"}, mode). It searches PATH and passes each argument unchanged; spaces and shell metacharacters are ordinary argument bytes. The Zsh mode is not accepted by this factory. command() returns the executable name supplied as argv[0].

For a complete command result in one call, include <mc/system.h> and link a compiled Catalyst library:

auto result = mc::cRun({"git", "status", "--short"});
std::cout << result.out;
std::cerr << result.err;
if(result.status != 0){ /* command failed */ }

auto copied = mc::cRun({"/bin/cat"}, "input bytes\n");
auto pipeline = mc::cRunShell("printf hello | tr a-z A-Z");

cRun(arguments, input = "") captures both streams while sending input followed by EOF. It waits for completion and returns CRunResult{status, out, err}, retaining embedded NUL bytes. Nonzero exit status is returned, while launch or I/O failures throw CError. Signal termination uses the same status encoding as CCommand below. cRunShell(command, input = "") explicitly runs /bin/sh -c without login or interactive startup files. cFindExecutable(name) searches PATH and returns an absolute executable path, or an empty string if none is found. These convenience calls capture the entire output in memory and have no timeout; use CCommand for incremental output and lifecycle control.

Combine modes with |. Streams that are not redirected inherit the application's corresponding standard streams.

mc::CCommand command("exec /bin/cat",
                     mc::CCommand::Input | mc::CCommand::Output);
command.write("first line\nsecond line\n");
command.closeInput();             // Send EOF; safe to call again.
mc::cstr output;
command.readOutput(output);       // Append all output through EOF.
int result = command.await();

write() waits until the entire string has been written or reports CError. Strings may contain binary data, including NUL bytes. A closed input pipe reports an error without terminating the application with SIGPIPE. A failed write may already have delivered a prefix; do not assume it can safely be retried in full.

The one-argument readOutput(text) and readError(text) append bytes through EOF. Their (text, timeout) overloads append currently available bytes, waiting up to the given number of seconds for data or EOF. They return false on timeout and true on data or EOF; true with no appended bytes means EOF. A zero timeout polls. Timeouts must be finite, nonnegative, and representable.

matchOutput(regex, matches, timeout) and matchError(regex, matches, timeout) attempt a full CRegex match against the unread stream, appending the whole match and capture groups to matches. For a suffix match, include a prefix such as [\s\S]* in the pattern. A successful match consumes the matched bytes. Timeout or EOF without a match returns false and leaves the bytes available for another match or read. Each call uses one total waiting budget; new data does not restart it. Regular-expression evaluation itself is not interruptible.

Both captured streams continue to drain during reads, writes, and await(), so filling one pipe does not prevent reading the other. Captured bytes stay in memory until consumed. Read incrementally for large or continuous output. Methods on a live command can be called from different threads; reads of the same stream serialize. Closing input, closing the command, or calling await() cancels a concurrent unfinished write.

Command status and lifetime

await() closes standard input, waits for the immediate child to exit and captured streams to reach EOF, and returns the exit status. Output remains available afterward. Descendants that inherit captured streams can delay EOF; use close() to stop waiting for their output. An ordinary nonzero shell exit is a result, not an exception.

status() polls without waiting. Completed results are cached, so repeated status() and await() calls agree. processId() returns the owned child PID, or -1 after it has been reaped. Let CCommand own child waiting: do not call waitpid() for its PID or install a competing global child reaper. It requires waitable children and rejects SIGCHLD settings that automatically discard status.

signal(number) signals the immediate child and is harmless after completion. close(true) closes redirected streams, signals the command with SIGTERM, and waits for cleanup. setCloseSignal(number) changes that first signal. Cleanup escalates to SIGKILL after one second, or when the group leader exits. close(false) starts the same cleanup and returns immediately; the child is still reaped. Both close forms are safe to repeat. Processes that leave the group, or outlive an already-reaped leader, are outside this ownership.

Commands with piped input, or whose inherited input is not a terminal, have their own process group; closing them also signals that group. Commands that inherit terminal input stay in the terminal's group so they can read it normally. In that case, closing targets the immediate child. Select Input when you want to supply input yourself and include subprocesses in group cleanup.

A command cannot be copied or moved. Destroying a normal command performs the equivalent of close(true). CResourceManager::shutdown() also closes normal commands and can be called repeatedly. A Persistent command is skipped by that shutdown; destroying it sends EOF on redirected input and discards subsequent captured output while allowing it to finish. Explicit close() still stops a persistent command.

Finish application calls on an object before destroying it. System failures and invalid arguments report CError; destruction completes cleanup without propagating command errors.

Run commands across machines with Nexus

mc-nexus runs shell commands concurrently on SSH destinations selected from named groups. It supports command aliases, multiple instances per machine, collective logging, and cancellation of the current batch. It runs on macOS and Linux and reads commands either interactively, with line editing and session history, or from standard input.

Start Nexus

Nexus and its helper are included in the SDK's bin directory. After configuring your environment, customize conf/mc-nexus.ccfg for your own machines and commands, then launch:

mc-nexus

The SDK also provides bin/nexus as a relative symbolic link to mc-nexus, so you can launch the command as nexus.

Configure SSH keys or an agent and verify the destination host keys before using Nexus. Connections use SSH batch mode: password and host-key confirmation prompts are unavailable. Remote machines need Bash 3.2 or newer. Nexus supplies the helper when it connects, so remote machines do not need Catalyst, mc-run, or mc-nexus-run installed. Commands run through a Bash login shell, including its startup files. They have no interactive terminal and receive EOF on standard input.

Configuration and commands

The supplied configuration is conf/mc-nexus.ccfg. Nexus uses CProgram's configuration order: command-line options, mc-nexus.ccfg in the current directory, directories in CM_CONF_PATH, then $MC_HOME/conf/mc-nexus.ccfg. A minimal configuration is:

{
  directory: "~/project"
  group: "workers"
  groups: {workers: [worker1 worker2]}
  logging: true
  logPath: "$(HOME)/log/nexus"
  commands: {
    build: {command: "cmake --build build --config Release"}
    worker: {command: "./worker", instanced: true, threaded: true}
  }
  instances: {worker1: 2, worker2: 1}
  threads: {worker1: 4, worker2: 8}
}

Use quoted strings for top-level string settings such as group and directory. Machine names inside groups can be strings or CSON symbols. A destination may also be an SSH alias or user@host. Every group must contain distinct destinations. A nonempty machine setting selects a single destination and overrides group.

mc-nexus -group workers -logging -verbose

# At the Nexus prompt:
build
worker -port 5585!
machine = "worker1"
instances = {worker1: 3}
verbose = true
cd "build directory"
pwd
reconfigure
quit

An unrecognized command is sent as shell text to each selected machine, once per machine. An alias replaces its first word with the configured command and appends the remaining arguments. Aliases can override directory, group, and machine. Within alias command text, $host is replaced by the local Nexus machine's short hostname.

An alias with instanced: true uses the top-level instances map; otherwise it uses its own instances map, defaulting to one instance per machine. Counts must be nonnegative integers; zero skips that machine. Similarly, threaded: true selects the top-level threads map, while an alias's own threads map applies otherwise. A positive thread count appends -threads N to the command. An alias's args map appends additional shell text for each destination. Raw shell commands do not use these alias instance or thread settings.

reserve, defaulting to two, reduces an instance count only when the destination exactly matches the local short hostname and its requested count equals the local hardware thread count. It subtracts the reserve only when the count is larger than the reserve.

Assignments to existing settings take CSON values, validate them, and apply to subsequent commands. An invalid assignment leaves the previous configuration in use. cd and change update the remote working directory; cd alone selects the remote home directory. Relative paths are appended to the selected directory, and ~ or ~/path refers to each remote user's home. A failed directory change prevents the command from executing there. reconfigure reloads configuration while retaining the current remote directory. exit, quit, or EOF leaves Nexus. historyCount sets the in-memory history limit.

Output and logging

verbose: true displays output as it arrives. An unquoted, unescaped trailing ! enables this for one command. Output lines identify the destination, zero-based instance, and stream, for example worker1[0] out: ready. Without verbose output, failures still show a bounded tail of stderr. silent: true suppresses successful completion messages.

logging: true creates a fresh run-... directory below logPath for every batch. Each instance has separate host.instance.out and host.instance.err files containing its raw output. Special characters in destination names are encoded in filenames. collective.log contains labelled output and completion summaries from all instances. Its order reflects when Nexus observes output; it does not establish an exact event order across machines or between stdout and stderr.

Logs are written incrementally, including final output without a newline. Very long lines are split into chunks in the collective log and console; raw stream files retain the original bytes. Log files are opened before commands start, and write failures cancel the batch. Verbose output alone does not enable logging or overwrite earlier logs.

Interrupts and batch input

Ctrl-C cancels the current batch and returns to the interactive prompt. At an idle prompt it discards the unfinished command. Cancellation first sends SIGINT to each job, waits interruptGrace seconds (default two), then sends SIGTERM if needed and waits the same interval before forcing cleanup. Cancellation targets that job and its process group. Commands that deliberately detach into a different process group, such as daemons, are outside that cleanup.

SIGTERM, SIGHUP, and SIGQUIT cancel active work and exit Nexus. Loss of the SSH input connection also requests remote cancellation. If the connection stalls or cleanup cannot be confirmed, Nexus reports remote cleanup unconfirmed and stops waiting after a bounded interval. connectTimeout, default ten seconds, limits SSH connection establishment. Remote cleanup cannot be guaranteed during a network partition.

printf 'build!\n' | mc-nexus -group workers

Commands read from a pipe run one batch at a time. Exit status is zero when the last operation succeeds and one when it fails. Interrupting a batch from piped input exits with 130; termination by another handled signal exits with 128 plus its signal number. Use one command per invocation when its exit status must describe that command alone.

Build and run Cosmic programs

The mc-cosmic command compiles a Cosmic .cc file into an executable, an object file, or LLVM IR. Cosmic combines C++ types and expressions with conveniences such as indentation-based blocks, % for return, string interpolation, properties, and generated method forwarding. See Cosmic language features for the syntax and behavior of these extensions.

The SDK includes mc-cosmic. With the environment from SDK setup, compile and run a program directly. Keep Xcode or the Command Line Tools selected with xcode-select so the compiler can find Apple's SDK and linker:

mc-cosmic -g -o hello hello.cc
./hello

A complete minimal hello.cc is:

#include <mc/CProgram.h>
using namespace mc;

int main(int argc, char** argv)
  CProgram program(argc, argv)
  cvar answer = 42
  print answer
  %0

# Produce an object to link with other C++ objects or libraries.
mc-cosmic -c -g -o hello.o hello.cc

# Optimize an executable.
mc-cosmic -O2 -o hello hello.cc

# Write LLVM IR.
mc-cosmic --emit-llvm -o hello.ll hello.cc

For programs that use the Mac graphics library, see Link the Mac graphics library. mc-cosmic links it automatically on macOS when producing an executable.

For integration with other build tools, combine -c with --depfile hello.d to write header dependencies. Repeat --clang-arg ARG to pass C++ compiler flags, for example --clang-arg -fPIC for a shared-library object. The SDK's CMake integration supplies the relevant target options automatically.

The compiler accepts one input file per invocation. Without -o, output uses the input's base name in the current directory, with the appropriate extension for object, IR, or Metal output. Failed compilation preserves an existing output file. Repeat -I DIR to add include directories and -D NAME=VALUE to define macros used by Cosmic source and ordinary C++ headers. Such headers can provide classes, functions, and templates used from Cosmic.

Quoted and angle-bracket includes, such as #include "cx/Plot.hpp" and #include <Metal/Metal.hpp>, contain ordinary C++. These headers retain ordinary C++ syntax and formatting when used from Cosmic. This also applies to .h and .hxx headers. No special directive is needed for .hpp.

A C++ #include takes effect where it appears in the Cosmic source. The included code can use preceding declarations and definitions in its enclosing scope, along with the current macro definitions. Fragments may introduce namespace declarations, class members, or statements inside a function. Indent directives inside an indented function body just like its other statements.

Use #define and #undef to configure subsequent source and includes. Macro changes made by headers remain available to later source. Header guards and #pragma once retain their C++ behavior; an unguarded header can be included repeatedly with different macro values:

#define VALUE_TYPE u1
#define INDEX_TYPE u1
#define ROW_TYPE u4
#include "table_index.h"
#undef ROW_TYPE
#define ROW_TYPE u8
#include "table_index.h"
#undef ROW_TYPE
#undef VALUE_TYPE
#undef INDEX_TYPE

Diagnostics and debug builds retain the original Cosmic and included C++ filenames and line numbers. Dependency files include the fragments used during compilation.

Cosmic preprocesses source before parsing declarations, statements, and expressions. This supports object and function macros, stringification, token pasting, variadic macros with __VA_OPT__, and macro include names. Use #if, #ifdef, #ifndef, #elif, #elifdef, and #elifndef to select source; inactive branches are not parsed. #error, #warning, #line, and _Pragma have their C++23 behavior. Diagnostics and debug information honor logical filenames and lines set by #line.

Expanded source still follows Cosmic's spacing and newline rules. Backslash continuations splice physical lines before those rules apply. Macros from ordinary C++ headers retain C++ operator, lambda, and literal meanings. Tokens written in Cosmic source, including macro arguments, retain Cosmic conventions. Unknown pragmas are ignored. Supported pragmas include header guards with once, macro stacks with push_macro and pop_macro, and declaration packing with pack.

// example.cc
export module example;
export int answer()
  return 42

// consumer.cc
import example;
int main()
  return answer() == 42 ? 0 : 1

mc-cosmic --precompile example.cc -o example.pcm
mc-cosmic -c example.cc -o example.o
mc-cosmic --clang-arg -fprebuilt-module-path=. \
  --link example.o consumer.cc -o consumer

cmake --build examples-build --target example-cosmic-messages
./examples-build/example-cosmic-messages

int absolute(int value)
  if value < 0
    %-value
  else
    %value

int absoluteWithBraces(int value){
  if(value < 0){ return -value; }
  return value;
}

int add(int x, int y) x + y

[i4] values = {1 2 3}
int total = add(20 22)
cvar settings = {name: "Ada" retries: 3}

int absoluteWithBraces(int value)
{
  if(value < 0)
  {
    return -value;
  }
  return value;
}

namespace demo{

struct Item{
  int value = 0
end Item

end namespace demo

#include core CBuffer thread

Ints => [int]
Counts => {cstr: int}

void example(cstr# name)
  Ints values = {1, 2, 3}
  Counts counts = {{"ready", 3}, {"waiting", 1}}
  cstr# label = name
  c1% text = "read only"

Values => [f4]                 // using Values = CVector<f4>
f8 Pi => 3.141592653589793     // static constexpr double

[f8] values = {1.0 2.0 3.0}  // CVector<f8>
{cstr} flags = {"ready", "local"} // CHashSet<cstr>
{cstr:u8} countMap = {{"ready" 2}} // CHashMap<cstr, u8>
{cstr:cvar} settings = {{"enabled" true}}
// With a declared class Foo, [Foo] is CVector<Foo>.

count := 42
name := "example"
values := CVector<int>{1, 2, 3}
alias :=> values
alias[0] = 9
// values[0] is now 9.

x := 0
[f8] values = {1.0 2.0}
copy := values[0]
v :=> values[0]
v = 9.0
// copy is still 1.0; values[0] is now 9.0.

int twice(int value) value * 2

struct Counter{
  int value_ = 0

  int value()# value_
  setValue(int value) value_ = value

  reset()
    value_ = 0
    %void
}

struct Foo{
  f4 size_ = 2.0f
  f4 size()# size_
}

void showLabel(cstr# s){ print s; }

generic<T>
T twice(T value) value * 2

int foo(int x)
  if x > 2 %x * 2
  %x

struct Point{
  f8 x_
  f8 y_

  Point(f8 x, f8 y)
  : x_ = x
  y_ = y
    assert x_ >= 0
}

generic<T>
struct Box{
  T value
  T get()# value
}

generic<T>
T twice(T value) value + value

generic<T>
Values => [T]

cvar result = {answer: 42}
cstr message = "answer ${result.at(\"answer\")}"
cstr joined = "first " "second"
cstr lines = """first line
second line"""

#include core

CRegex words("\b[A-Za-z]+\s+[A-Za-z]+\b")
bool matched = words.match("Ada Lovelace")
cstr escapes = "\b\s"
// escapes contains the four bytes '\\', 'b', '\\', 's'.
// Its C++ spellings are "\\b\\s" or R"(\b\s)".

cvec labels = {"fast", "local"}
cvar options = {
  #: "job"
  retries: 3
  labels: labels
  nested: {enabled: true}
}
options:retries = 4
cvar retries = options:retries|10
bool hasRetries = options?retries

cvar message = {
  name: "Ada"
  scores: [10 20 30]
  nested: {enabled: true}
}
message:status = "ready"         // message["status"] = "ready"
cvar name = message:name         // message.at("name")
cvar retries = message:retries|3 // message.get("retries", 3)
// The default does not insert a retries field.

[i4] values = {10 20}
values.0 = 42
std::tuple<i4, cstr> pair{7, "seven"}
int first = pair.0
// values[0] is now 42; first is 7.

i4 x = 20
i4 y = 22
csym symbol("x")          // explicit equivalent of `x
cfunc request("Foo", x y) // explicit equivalent of `Foo(x y)
// symbol names "x"; request names "Foo" with arguments 20 and 22.

Counter counter
counter`value = 7
int result = counter`value
Counter* pointer = &counter
pointer->`value = result + 1

struct Base{ virtual ~Base() = default; }
struct Test : Base{ i4 value = 42; }

Test test
Base* y = &test
checked := y as? Test*       // checked downcast; failure would be 0p
known := y as Test*          // valid because y really points to a Test
bytes := &test as! u1*       // reinterpret the address
restored := bytes as! Test*  // round-trip to the original pointer type
const Test* view = &test
mutableView := view as# Test*
mutableView->value = 53      // test was originally mutable

cstr source = "hello"
cstr destination = &&source
source = "reused" // valid; the old contents of source are not promised

#include time
now := .now() // auto now = cNow()

#include core

int main(int argc, char** argv)
  .option("name:n", "Cosmic", "Name to greet")
  CProgram program(argc, argv)
  cvar# args = .args()
  print args["name"]
  %0

f8 x = 2.0
f8 y = 3.0
f8 power = x^y // 8.0
i4 bits = 2 ^ 3 // 1: spaced ^ is C++ bitwise XOR

int total = 0
for 4
  total += $
// total is 0 + 1 + 2 + 3.

for(i: 2, 5)
  total += i
// Adds 2 + 3 + 4.

i4 total = 0
for(i : 10){ total += i; }
// total is 45; i visits 0 through 9.

CVector<int> values = {1, 2, 3}
values|
  $ *= 2
values|total += $

{cstr: int} scores = {{"first", 3}, {"second", 4}}
scores|
  print "${$k}: ${$v}"
  $v += 1
scores|print "${$k}:${$v}"

for(cstr# key; int& value : scores)
  value += 1
  print "${key}: ${value}"

i4 attempts = 0
i4 maxAttempts = 3
forever{
  if attempts >= maxAttempts break;
  ++attempts;
}

[i4] values = {1 2 3}
while !values.empty() values.pop_back();

auto twice = []{ %$ * 2 }
auto answer = [](){ %42 }
int value = 21
int result = twice(value)

#include <algorithm>

CVector<int> values = {1, 2, 3}
int total = 0
std::for_each(values.begin(), values.end()){
  total += $0;
}

[i4] values = {3 1 2}
std::sort(values.begin(), values.end()){
  return $0 < $1;
};
// values is now {1, 2, 3}.

void send(cvar& received, cvar# message){
  received = message;
}

cvar received
send(received){
  message: "hello"
};
// Calls send(received, cvar{...}); received now has a message field.

int result = 0
dispatch 2
  case 1
    result = 10
  case 2
    result = 20
  default
    result = -1
// result is 20; execution does not continue into default.

cstr command = "open"
int result = 0
dispatch cHash(command)
  case `open
    result = 1
  case `close
    result = 2
  default
    result = -1

cfunc request("scale", 7, 6)
dispatch request
  case "scale"2#
    print request[0] * request[1]
  default
    error "unknown operation"

csym command("open")
dispatch command
  case `open
    print "opening"
  default
    print "unknown command"

cfunc request("Foo", 20 22)
dispatch request
  case "Foo"1#
    print "one argument"
  case "Foo"2#
    print request[0] + request[1]
  default
    print "unknown function"
// The second dispatch prints 42.

#include core

class Events: public CHandler{
public:
  int total = 0
handlers:
  add(cvar& message)
    total += message:amount.getInt()
}

// Inside a function:
Events events
cvar message = {#: "add", amount: 42}
bool handled = events.handle(message)

class Calculator: public CExecutor{
executors:
  int scale(int value, int factor = 2) value * factor
  clear(){}
}

// Inside a function:
Calculator calculator
CExecutor& executor = calculator
cfunc request("scale", 21)
if executor.canExecute(request)
  cvar answer = executor.execute(request)

struct Counter{
  int value = 0

  defer int increment()
    %++value
}

#include <mc/CBuffer.h>

storable struct Stored{
  int first
  int second
  Stored(int a, int b) : first(a), second(b){}
}

// Inside a function:
Stored original(12, 30)
CBuffer buffer
original.store(buffer)
buffer.rewind()
Stored restored(buffer)

// In Counter_, with int add(int) declared by Counter:
outer_methods:
  int add(int amount)
    value_ += amount
    %value_

#include <mc/thread.h>

int takeNext(CMutex& mutex, int& next)
  guard(mutex)
    %next++

CMutex mutex
i4 i = 0
guard(mutex){
  ++i;
}
guard mutex ++i;
guard mutex
  ++i
// i is now 3; each guard has already released the mutex.

cstr name = "Ada"
print "hello ${name}"     // newline and flush are added
output "hello ${name}\n" // newline is explicit
log 22                  // diagnostic output includes source information
i4 x = 3
assert x > 2

try
  error "an error occurred"
catch(CError& failure)
  print failure.what()

// Person is generated from the adjacent database.cson schema.
Person person{name: "Fred", score: 53.0}
// The same syntax with an age field in the schema is:
// Person person{age: 53, name: "Fred"}

Person ada{id: 1, name: "Ada", score: 42.0,
  notes: {language: "Cosmic"}}

int kindCount()
  s8 First => CUniqueId("localKinds")
  s8 Second => CUniqueId("localKinds")
  s8 Count => CUniqueCount("localKinds")
  %Count

Database rows and queries

Cosmic can generate typed rows for CDatabase from a CSON schema. Begin the map with #type: database to enable automatic generation. Each remaining top-level key names a row class and its table. The fields vector specifies the fields in their storage order; indices lists indexed fields or composite indices:

{
  #type: database
  Item: {
    fields: [
      {#: id type: u4 unique: true}
      {#: group type: u4}
      {#: rank type: f8}
      {#: name type: str optional: true}
      {#: payload type: var optional: true}
    ]
    indices: [id rank [rank group]]
  }
}

Place the schema beside the source files that use it. In a CMake application, call catalyst_cosmic_directory("${CMAKE_CURRENT_SOURCE_DIR}") after defining the targets. The SDK's integration generates matching headers in the build directory and supplies their include paths. For example, examples/14-database/database.cson produces database.hxx; include it with #include "database.hxx" from either C++ or Cosmic.

Multiple schemas in a directory are supported: users.cson produces users.hxx, and orders.cson produces orders.hxx. The marker must be the first field in the map; quoted "#type": "database" also works. Other CSON files are left alone. New schemas and schema edits are picked up on the next build. A failed generation stops the build and preserves the last valid header.

Keep the CSON file as the source of truth. Remove any old generated .hxx with the same name from the source directory when adopting automatic generation, so quoted includes cannot select a stale copy. Each build directory owns its generated headers; do not edit or commit them.

For manual generation without CMake:

mc-cosmic --database -o app_database.hxx app_database.cson

The original -database spelling also works. Without -o, this example writes app_database.hxx in the current directory. Explicit --database also accepts older schemas without the marker. Generation reports invalid types, names, and index definitions, and preserves an existing output on failure.

Field types are b1 (Boolean), u1/i1/u2/i2/u4/i4/u8/i8 (unsigned/signed integers with the indicated byte width), f2/f4/f8 (floating point), str, and var. Rows use 32-bit row IDs by default; add big: true to a table for 64-bit row IDs. The database assigns row IDs separately from application fields such as id.

unique: true rejects duplicate indexed values. optional: true permits an indexed field to be absent. hash: true retains the index hash while removing that field's original value from the stored row. Preserve field order, types, and index definitions when reopening an existing table.

Include CDatabase before the generated header. The header contains row declarations and can be included inside a namespace. Create or open the actual table with the generated type:

#include core CDatabase
#include "app_database.hxx"

// Inside a program with a live CProgram and CDatabase:
auto table = database.createTable<Item>()
Item row = {id: 42, group: 7, rank: 3.5, name: "example"}
row`payload = cvec{1, true, "data"}
table->insert(&&row)
table->commit()

Item found
if table:get(found; Id(42); name payload)
  std::cout << found`name << '\n'

A field named name provides Name, setName(value), name(), and hasName(). Backtick access uses these getters and setters. A local row's map initializer assigns its named fields through setters; the map does not specify an evaluation order for field expressions. Strings and cvars have both copying and moving setters. The generated Fields and Indices aliases remain available for direct use with the CDatabase C++ interface.

table:get(row; Id(value)) returns whether it found a row through the Id index. Use RowId(value) to fetch by the database-assigned row ID. An optional final semicolon introduces the fields to load, separated by spaces or commas. Omit that list to load every field. Unselected fields are absent in the returned row; the database-assigned row ID remains available.

table:forward(){
  std::cout << $`name << '\n'
}
table:backward(Rank; name){
  std::cout << $`name << '\n'
  %false
}
table:query(Rank(2.0, 5.0); name rank){
  std::cout << $`name << " " << $`rank << '\n'
}

Without an index, forward() and backward() traverse by row ID. For projection without an index, write forward(name payload). An index name selects index order; query(Index(value)) selects an equal value, and a numeric index also accepts an inclusive start/end range. The table receiver can be a smart pointer, a raw pointer, or a reference to either.

Inside a traversal, $ is a reference to the current row. Reaching the end of the callback continues traversal; %false stops, and %true continues immediately. A return exits the callback, not the enclosing function. Update or erase rows through the table, then commit after traversal:

table:query(Id(42)){
  $`name = "changed"
  table->update($)
}
table->commit()

table:query(Id(42)){ table->erase($) }
table->commit()

A composite index such as [rank group] provides the name RankGroup. Its first field supplies ordering; the remaining fields select the group. For example, table:get(row; RankGroup(3.5, u4(7))) finds that rank in group 7, and table:forward(RankGroup(u4(7)); name){ ... } traverses that group's rows. These operations follow the same CDatabase commit and persistence rules as direct C++ calls. Generate the schema header before compiling a source that includes it; use -I if it is in another directory.

For a named callback passed to the table's C++ methods, the original cquery form is also available. Its named row parameter is a reference, and its body must return a Boolean on every path:

auto callback = cquery(Item, row){
  std::cout << row`name << '\n'
  %true
}
table->traverseForward(callback)

The compiler locates its bundled resources, headers, and libraries relative to the SDK. Keep that directory layout intact. To select locations explicitly, use --resources DIR, --sdk-include DIR, --library FILE, or --sysroot DIR. This release supports macOS on Apple Silicon; Linux support is coming soon.

Cosmic GPU programs

Mark GPU functions with gpu and entry points with kernel. hybrid retains a declaration for both CPU and GPU use where its body is valid in both environments. GPU entry points have access to conveniences such as ThreadId. For example:

gpu float squared(float x)
  %x * x

kernel void fill(float* values)
  values[ThreadId] = squared(2.0f)

mc-cosmic --emit-metal \
  -o kernels.metal kernels.cc
mc-cosmic --metallib \
  -o kernels.metallib kernels.cc

Metal source generation uses the supplied Cosmic GPU support library. Building a .metallib requires macOS, Xcode, and Apple's Metal toolchain. If Xcode reports that the component is missing, install it with xcodebuild -downloadComponent MetalToolchain. Generated Metal source retains original file and line directives.

GPU vector aliases such as f4x4 select Metal's vector types directly. Available element types and lane counts depend on the Metal toolchain and target. An alias for a reserved wide or double vector type does not enable that unsupported type.

The runnable tutorial in examples/27-compute/main.cc defines a gpu helper, a kernel entry point, and its CPU main() in one Cosmic file. Its CMake target compiles the GPU declarations with --metallib and the host code as an ordinary executable. CGLRun loads the resulting library, binds the host vector, dispatches the kernel, and copies the results back. After configuring the tutorial project, build and run it on macOS with:

cmake --build examples-build --target example-compute
./examples-build/example-compute

Compile the CPU executable and GPU library separately. Existing CGL_ run/interact calls are retained. The application must supply the CGL_ declarations and implementation, including its library-loading and dispatch behavior; this compiler does not provide that runtime or automatically load a generated Metal library. Use object output and your normal C++ link command when additional application libraries are needed.

Debug Cosmic source

Compile with -g -O0 to keep source locations and local variables. Breakpoints and stack frames refer to the original Cosmic .cc source. On macOS the compiler creates an adjacent .dSYM bundle for an executable; keep that bundle and the matching source files available.

lldb ./hello
(lldb) breakpoint set --file hello.cc --line 6
(lldb) run
(lldb) next
(lldb) bt
(lldb) frame variable

Object files produced with -c -g carry their own debugging information. Optimized builds can inline calls and remove variables. Launching a program under the debugger requires the host's debugger permissions.

Build and run Microcosm programs

The mc-microcosm command compiles Microcosm source into a native executable or object file. It supports classes, constructors, methods, default arguments, recursive functions, collections, references, string interpolation, conditionals, and loops, subject to the limits below.

The compiler is included in the SDK. After configuring your environment, save this small program as hello.mc:

import cms

values := [3, 5, 8]
total := 0
values|total += $
print "Total: ${total}"

mc-microcosm -g -O0 -o hello-microcosm hello.mc
./hello-microcosm

# Produce an object file instead of linking an executable.
mc-microcosm -g -c -o hello-microcosm.o hello.mc

Both the compiler and generated programs use Catalyst's CProgram conventions, including MC_HOME. Select optimization with one of -O0 through -O3; the default is -O0. Use --emit-llvm -o program.ll to write LLVM IR. Failed compilation preserves an existing output file.

The compiler locates its headers and libraries in the SDK. Override resource selection with --sdk-include DIR, --library FILE, and --sysroot DIR.

Native compilation does not yet support lambdas, comprehensions, switch and try/catch statements, declaration modifiers, overloaded global functions, or imports other than cms. Unsupported constructs report errors. This release supports macOS on Apple Silicon; Linux support is coming soon.

Debug Microcosm source

Compile with -g -O0 to retain source locations and local variables with optimization disabled. Breakpoints and stack frames refer to the original .mc file and its line numbers, including expressions inside interpolated strings. Runtime helper frames may also appear when stepping into framework operations.

lldb ./hello-microcosm
(lldb) breakpoint set --file hello.mc --line 5
(lldb) run
(lldb) next
(lldb) step
(lldb) bt
(lldb) frame variable

On macOS, an executable built with -g gets a companion hello-microcosm.dSYM bundle. Keep it with the executable; the compiler's temporary object is not needed for debugging. Debug object files produced with -c -g retain their own DWARF information. The SDK includes the tool needed to produce these macOS debug bundles.

Keep the original source files available to the debugger. If they move, use LLDB's settings set target.source-map OLD_ROOT NEW_ROOT. Higher optimization levels can inline calls, remove variables, and make stepping less direct even with -g. Interactive stepping requires a host that permits debugger launches.

When Catalyst's error-stack capture is enabled, uncaught CError exceptions also use these source locations in stack traces. Failures inside native Microcosm functions then identify the original file and call-site lines.

Parse Microcosm source

mc::CMParser reads Microcosm programs and expressions into cvar syntax trees. Include <mc/CMParser.h> and link Catalyst::Shared. Parsing describes the program; it does not execute it.

mc::CMParser parser;
mc::cvar program = parser.parseFile("simulation.mc");
mc::cvar expression = parser.parseExpr("1 + 2 * 3");
// expression is Add(1, Mul(2, 3)).

parse(text) reads a complete program from a mc::cstr or a NUL-terminated string. The result is a Block whose first entry is Module(name) or PrivateModule(name). Declarations and statements follow in source order. parseExpr(text) returns one expression. Imports are retained in the tree; parsing does not require imported modules to be installed.

The parser accepts the existing indentation-based language, including classes, functions and default arguments, control flow, lambdas, collection literals, property access, and string interpolation such as "value ${a + 2}". Switch cases contain expression trees: Switch(condition, [Case(expression, block), …], defaultBlock). The default is none when absent.

Integer literals may use decimal, a leading zero for octal, 0b for binary, or 0x for hexadecimal. Invalid digits, missing digits after a prefix, and overflow are errors. Binary, octal, and hexadecimal literals can express all 64 bits; for example, 0xffffffffffffffff represents -1.

Function and constructor parameters must have distinct names. Once a parameter has a default value, every following parameter must also have one. Lambda arguments $0 through $14 are detected inside collection literals as well as ordinary expressions. Nested lambdas have their own arguments, and each lambda requires its closing }.

Map keys may use either single or double quotes. Block comments /* … */ may appear between expression tokens. Keywords match complete names, so a name such as elsewhere remains an ordinary identifier. Indentation determines which switch owns a case, including when switches are nested.

A parenthesized comma expression, such as (count += 1, count + 2), evaluates its left side first and returns the right side's result. It can be passed as a single function argument with f((a, b)); f(a, b) passes two arguments.

Use setFileId(id) before parsing when combining several files. Statements carry the file ID in tag().kind and the source line in tag().attrs. tokens() exposes the most recent parse's token offsets and categories. Copy tokens if you need them after the next parse. discardComments(false) retains standalone comments as syntax-tree entries.

Expression nodes also carry source lines where available. Multiline statements retain their starting line, and expressions inside ${…} refer to lines in the enclosing source file. An escaped \n inside a string does not advance the source line. parseExpr() rejects additional statements following the expression.

Malformed input reports CParseError, including a line number and, for parseFile(), the path. A parser can be reused after either success or failure. Each parse starts a new module. Source files are limited to 16,777,215 bytes and individual tokens to 65,535 bytes. Excessive nesting and numeric literals outside their supported range are rejected. Use separate parser objects concurrently.

Microcosm values and helpers

Microcosm uses Catalyst values for scalars, collections, expressions, objects, and callable values. Its core helpers include printing, string conversion, mathematics, filesystem operations, environment variables, program options, random numbers, and calendar conversions.

Calendar functions that fill several outputs require references to variables. Pass each output with &, as in .getDayInfo(2024 2 29 &dayOfYear &dayOfWeek). Invalid output arguments are rejected before changing any outputs. Epoch conversions use the same local-time conventions as Catalyst's date and time functions.

Distance requires vectors of equal length. Character predicates return false for an empty string and otherwise examine its first byte. A loop over an integer count visits the indices from zero through count - 1; a nonpositive count visits no indices.

<mc/CMInterpreter.h> supplies the CMInterpreter<I> base for custom interpreters. Get(), Put(), and Push() retain references when operating on interpreter variables; those references remain valid only while their targets exist. Results computed from temporary containers own their values. Its host-side cGridSize() helper currently throws because no GPU execution grid is available to the interpreter.