Package Exports

This package does not declare an exports field, so the exports above have been automatically detected and optimized by JSPM instead. If any package subpath is missing, it is recommended to post an issue to the original package (@memberjunction/core) to support the "exports" field. If that is not possible, create a JSPM override to customize the exports field for this package.

Readme

@memberjunction/core

The @memberjunction/core library provides a comprehensive interface for accessing and managing metadata within MemberJunction, along with facilities for working with entities, applications, and various other aspects central to the MemberJunction ecosystem. This library serves as the foundation for all MemberJunction applications and provides essential functionality for data access, manipulation, and metadata management.

Installation

npm install @memberjunction/core

Overview

The @memberjunction/core library is the central package in the MemberJunction ecosystem, providing:

Metadata Management: Complete access to MemberJunction metadata including entities, fields, relationships, permissions, and more
Entity Data Access: Base classes and utilities for loading, saving, and manipulating entity records
View Execution: Powerful view running capabilities for both stored and dynamic views
Query & Report Execution: Tools for running queries and reports
Security & Authorization: User authentication, role management, and permission handling
Transaction Management: Support for grouped database transactions
Provider Architecture: Flexible provider model supporting different execution environments

Core Components

Metadata Class

The Metadata class is the primary entry point for accessing MemberJunction metadata and instantiating entity objects.

import { Metadata } from '@memberjunction/core';

// Create metadata instance
const md = new Metadata();

// Refresh cached metadata
await md.Refresh();

// Access various metadata collections
const applications = md.Applications;
const entities = md.Entities;
const currentUser = md.CurrentUser;
const roles = md.Roles;
const authorizations = md.Authorizations;

Key Metadata Properties

Applications: Array of all applications in the system
Entities: Array of all entity definitions
CurrentUser: Current authenticated user (when available)
Roles: System roles
AuditLogTypes: Available audit log types
Authorizations: Authorization definitions
Libraries: Registered libraries
Queries: Query definitions
QueryFields: Query field metadata
QueryCategories: Query categorization
QueryPermissions: Query-level permissions
VisibleExplorerNavigationItems: Navigation items visible to current user
AllExplorerNavigationItems: All navigation items (including hidden)

Helper Methods

// Get entity ID from name
const entityId = md.EntityIDFromName('Users');

// Get entity name from ID
const entityName = md.EntityNameFromID('12345');

// Find entity by name (case-insensitive)
const userEntity = md.EntityByName('users');

// Find entity by ID
const entity = md.EntityByID('12345');

// Get entity object instance (IMPORTANT: Always use this pattern)
const user = await md.GetEntityObject<UserEntity>('Users');

// NEW in v2.58.0: Load existing records directly with GetEntityObject
const existingUser = await md.GetEntityObject<UserEntity>('Users', CompositeKey.FromID(userId));

GetEntityObject() - Enhanced in v2.58.0

The GetEntityObject() method now supports overloaded signatures for both creating new records and loading existing ones in a single call:

Creating New Records

// Traditional approach - still supported
const user = await md.GetEntityObject<UserEntity>('Users');
user.NewRecord(); // No longer needed - done automatically!

// v2.58.0+ - NewRecord() is called automatically
const newUser = await md.GetEntityObject<UserEntity>('Users');
newUser.FirstName = 'John';
await newUser.Save();

Loading Existing Records (NEW)

// Traditional approach - still supported
const user = await md.GetEntityObject<UserEntity>('Users');
await user.Load(userId);

// v2.58.0+ - Load in a single call using CompositeKey
const existingUser = await md.GetEntityObject<UserEntity>('Users', CompositeKey.FromID(userId));

// Load by custom field
const userByEmail = await md.GetEntityObject<UserEntity>('Users', 
    CompositeKey.FromKeyValuePair('Email', 'user@example.com'));

// Load with composite primary key
const orderItem = await md.GetEntityObject<OrderItemEntity>('OrderItems',
    CompositeKey.FromKeyValuePairs([
        { FieldName: 'OrderID', Value: orderId },
        { FieldName: 'ProductID', Value: productId }
    ]));

// Server-side with context user
const order = await md.GetEntityObject<OrderEntity>('Orders', 
    CompositeKey.FromID(orderId), contextUser);

Automatic UUID Generation (NEW in v2.58.0)

For entities with non-auto-increment uniqueidentifier primary keys, UUIDs are now automatically generated when calling NewRecord():

// UUID is automatically generated for eligible entities
const action = await md.GetEntityObject<ActionEntity>('Actions');
console.log(action.ID); // '550e8400-e29b-41d4-a716-446655440000' (auto-generated)

BaseEntity Class

The BaseEntity class is the foundation for all entity record manipulation in MemberJunction. All entity classes generated by the MemberJunction code generator extend this class.

Loading Data from Objects (v2.52.0+)

The LoadFromData() method is now async to support subclasses that need to perform additional loading operations:

// Loading data from a plain object (now async)
const userData = {
    ID: '123',
    FirstName: 'Jane',
    LastName: 'Doe',
    Email: 'jane@example.com'
};

// LoadFromData is now async as of v2.52.0
await user.LoadFromData(userData);

// This change enables subclasses to perform async operations
// Example subclass implementation:
class ExtendedUserEntity extends UserEntity {
    public override async LoadFromData(data: any): Promise<boolean> {
        const result = await super.LoadFromData(data);
        if (result) {
            // Can now perform async operations
            await this.LoadUserPreferences();
            await this.LoadUserRoles();
        }
        return result;
    }
    
    // Important: Also override Load() for consistency
    public override async Load(ID: string): Promise<boolean> {
        const result = await super.Load(ID);
        if (result) {
            await this.LoadUserPreferences();
            await this.LoadUserRoles();
        }
        return result;
    }
}

Important: Subclasses that perform additional loading should override BOTH LoadFromData() and Load() methods to ensure consistent behavior regardless of how the entity is populated.

Entity Fields

Each entity field is represented by an EntityField object that tracks value, dirty state, and metadata:

// Access field value
const firstName = user.Get('FirstName');

// Set field value
user.Set('FirstName', 'Jane');

// Check if field is dirty
const isDirty = user.Fields.find(f => f.Name === 'FirstName').Dirty;

// Access field metadata
const field = user.Fields.find(f => f.Name === 'Email');
console.log(field.IsUnique); // true/false
console.log(field.IsPrimaryKey); // true/false
console.log(field.ReadOnly); // true/false

Working with BaseEntity and Spread Operator

IMPORTANT: BaseEntity uses TypeScript getter/setter properties for all entity fields. This means the JavaScript spread operator (...) will NOT capture entity field values because getters are not enumerable properties.

// ❌ WRONG - Spread operator doesn't capture getter properties
const userData = {
  ...userEntity,  // This will NOT include ID, FirstName, LastName, etc.
  customField: 'value'
};

// ✅ CORRECT - Use GetAll() to get plain object with all field values
const userData = {
  ...userEntity.GetAll(),  // Returns { ID: '...', FirstName: '...', LastName: '...', etc. }
  customField: 'value'
};

// ✅ ALSO CORRECT - Access properties individually
const userData = {
  ID: userEntity.ID,
  FirstName: userEntity.FirstName,
  LastName: userEntity.LastName,
  customField: 'value'
};

The GetAll() method returns a plain JavaScript object containing all entity field values, which can be safely used with the spread operator. This design choice enables:

Clean property access syntax (entity.Name vs entity.getName())
Full TypeScript/IntelliSense support
Easy property overriding in subclasses
Proper encapsulation with validation and side effects

Save Options

import { EntitySaveOptions } from '@memberjunction/core';

const options = new EntitySaveOptions();
options.IgnoreDirtyState = true; // Force save even if no changes detected
options.SkipEntityAIActions = true; // Skip AI-related actions
options.SkipEntityActions = true; // Skip entity actions
options.SkipOldValuesCheck = true; // Skip concurrency check (client-side only)

await entity.Save(options);

RunView Class

The RunView class provides powerful view execution capabilities for both stored views and dynamic queries.

import { RunView, RunViewParams } from '@memberjunction/core';

const rv = new RunView();

// Run a stored view by name
const params: RunViewParams = {
    ViewName: 'Active Users',
    ExtraFilter: 'CreatedDate > \'2024-01-01\'',
    UserSearchString: 'john'
};

const results = await rv.RunView(params);

// Run a dynamic view with entity objects returned
const dynamicResults = await rv.RunView<UserEntity>({
    EntityName: 'Users',
    ExtraFilter: 'IsActive = 1',
    OrderBy: 'LastName ASC, FirstName ASC',
    Fields: ['ID', 'FirstName', 'LastName', 'Email'],
    ResultType: 'entity_object' // Returns actual entity objects
});

// Access typed results
const users = dynamicResults.Results; // Properly typed as UserEntity[]

RunView Parameters

ViewID: ID of stored view to run
ViewName: Name of stored view to run
ViewEntity: Pre-loaded view entity object (optimal for performance)
EntityName: Entity name for dynamic views
ExtraFilter: Additional SQL WHERE clause
OrderBy: SQL ORDER BY clause (overrides stored view sorting)
Fields: Array of field names to return
UserSearchString: User search term
ExcludeUserViewRunID: Exclude records from specific prior run
ExcludeDataFromAllPriorViewRuns: Exclude all previously returned records
SaveViewResults: Store run results for future exclusion
IgnoreMaxRows: Bypass entity MaxRows setting
MaxRows: Maximum rows to return
StartRow: Row offset for pagination
ResultType: 'simple' (default) or 'entity_object'

RunQuery Class

The RunQuery class provides secure execution of parameterized stored queries with advanced SQL injection protection and type-safe parameter handling.

Basic Usage

import { RunQuery, RunQueryParams } from '@memberjunction/core';

const rq = new RunQuery();

// Execute by Query ID
const params: RunQueryParams = {
    QueryID: '12345',
    Parameters: {
        StartDate: '2024-01-01',
        EndDate: '2024-12-31',
        Status: 'Active'
    }
};

const results = await rq.RunQuery(params);

// Execute by Query Name and Category Path
const namedParams: RunQueryParams = {
    QueryName: 'Monthly Sales Report',
    CategoryPath: '/Sales/',  // Hierarchical path notation
    Parameters: {
        Month: 12,
        Year: 2024,
        MinAmount: 1000
    }
};

const namedResults = await rq.RunQuery(namedParams);

Parameterized Queries

RunQuery supports powerful parameterized queries using Nunjucks templates with built-in SQL injection protection:

-- Example stored query in the database
SELECT 
    o.ID,
    o.OrderDate,
    o.TotalAmount,
    c.CustomerName
FROM Orders o
INNER JOIN Customers c ON o.CustomerID = c.ID
WHERE 
    o.OrderDate >= {{ startDate | sqlDate }} AND
    o.OrderDate <= {{ endDate | sqlDate }} AND
    o.Status IN {{ statusList | sqlIn }} AND
    o.TotalAmount >= {{ minAmount | sqlNumber }}
{% if includeCustomerInfo %}
    AND c.IsActive = {{ isActive | sqlBoolean }}
{% endif %}
ORDER BY {{ orderClause | sqlNoKeywordsExpression }}

SQL Security Filters

RunQuery includes comprehensive SQL filters to prevent injection attacks:

sqlString Filter

Safely escapes string values by doubling single quotes and wrapping in quotes:

-- Template
WHERE CustomerName = {{ name | sqlString }}

-- Input: "O'Brien"
-- Output: WHERE CustomerName = 'O''Brien'

sqlNumber Filter

Validates and formats numeric values:

-- Template
WHERE Amount >= {{ minAmount | sqlNumber }}

-- Input: "1000.50"
-- Output: WHERE Amount >= 1000.5

sqlDate Filter

Formats dates in ISO 8601 format:

-- Template
WHERE CreatedDate >= {{ startDate | sqlDate }}

-- Input: "2024-01-15"
-- Output: WHERE CreatedDate >= '2024-01-15T00:00:00.000Z'

sqlBoolean Filter

Converts boolean values to SQL bit representation:

-- Template
WHERE IsActive = {{ active | sqlBoolean }}

-- Input: true
-- Output: WHERE IsActive = 1

sqlIdentifier Filter

Safely formats SQL identifiers (table/column names):

-- Template
SELECT * FROM {{ tableName | sqlIdentifier }}

-- Input: "UserAccounts"
-- Output: SELECT * FROM [UserAccounts]

sqlIn Filter

Formats arrays for SQL IN clauses:

-- Template
WHERE Status IN {{ statusList | sqlIn }}

-- Input: ['Active', 'Pending', 'Review']
-- Output: WHERE Status IN ('Active', 'Pending', 'Review')

sqlNoKeywordsExpression Filter (NEW)

Validates SQL expressions by blocking dangerous keywords while allowing safe expressions:

-- Template
ORDER BY {{ orderClause | sqlNoKeywordsExpression }}

-- ✅ ALLOWED: "Revenue DESC, CreatedDate ASC"
-- ✅ ALLOWED: "SUM(Amount) DESC"
-- ✅ ALLOWED: "CASE WHEN Amount > 1000 THEN 1 ELSE 0 END"
-- ❌ BLOCKED: "Revenue; DROP TABLE Users"
-- ❌ BLOCKED: "Revenue UNION SELECT * FROM Secrets"

Parameter Types and Validation

Query parameters are defined in the QueryParameter entity with automatic validation:

// Example parameter definitions
{
    name: 'startDate',
    type: 'date',
    isRequired: true,
    description: 'Start date for filtering records',
    sampleValue: '2024-01-01'
},
{
    name: 'statusList',
    type: 'array',
    isRequired: false,
    defaultValue: '["Active", "Pending"]',
    description: 'List of allowed status values'
},
{
    name: 'minAmount',
    type: 'number',
    isRequired: true,
    description: 'Minimum amount threshold'
}

Query Permissions

Queries support role-based access control:

// Check if user can run a query (server-side or client-side)
const query = md.Provider.Queries.find(q => q.ID === queryId);
const canRun = query.UserCanRun(contextUser);
const hasPermission = query.UserHasRunPermissions(contextUser);

// Queries are only executable if:
// 1. User has required role permissions
// 2. Query status is 'Approved'

Advanced Features

Conditional SQL Blocks

Use Nunjucks conditionals for dynamic query structure:

SELECT 
    CustomerID,
    CustomerName,
    TotalOrders
    {% if includeRevenue %}
    , TotalRevenue
    {% endif %}
FROM CustomerSummary
WHERE CreatedDate >= {{ startDate | sqlDate }}
{% if filterByRegion %}
    AND Region = {{ region | sqlString }}
{% endif %}

Complex Parameter Examples

const complexParams: RunQueryParams = {
    QueryName: 'Advanced Sales Analysis',
    Parameters: {
        // Date range
        startDate: '2024-01-01',
        endDate: '2024-12-31',
        
        // Array parameters
        regions: ['North', 'South', 'East'],
        productCategories: [1, 2, 5, 8],
        
        // Boolean flags
        includeDiscounts: true,
        excludeReturns: false,
        
        // Numeric thresholds
        minOrderValue: 500.00,
        maxOrderValue: 10000.00,
        
        // Dynamic expressions (safely validated)
        orderBy: 'TotalRevenue DESC, CustomerName ASC',
        groupingExpression: 'Region, ProductCategory'
    }
};

Error Handling

RunQuery provides detailed error information:

const result = await rq.RunQuery(params);

if (!result.Success) {
    console.error('Query failed:', result.ErrorMessage);
    
    // Common error types:
    // - "Query not found"
    // - "User does not have permission to run this query"
    // - "Query is not in an approved status (current status: Pending)"
    // - "Parameter validation failed: Required parameter 'startDate' is missing"
    // - "Dangerous SQL keyword detected: DROP"
    // - "Template processing failed: Invalid date: 'not-a-date'"
} else {
    console.log('Query executed successfully');
    console.log('Rows returned:', result.RowCount);
    console.log('Execution time:', result.ExecutionTime, 'ms');
    console.log('Applied parameters:', result.AppliedParameters);
    
    // Process results
    result.Results.forEach(row => {
        console.log('Row data:', row);
    });
}

Query Categories

Organize queries using categories for better management:

// Query by category path
const categoryParams: RunQueryParams = {
    QueryName: 'Top Customers',
    CategoryPath: '/Sales Reports/',  // Hierarchical path notation
    Parameters: { limit: 10 }
};

// Query with category ID
const categoryIdParams: RunQueryParams = {
    QueryName: 'Revenue Trends',
    CategoryID: 'sales-cat-123',
    Parameters: { months: 12 }
};

Best Practices for RunQuery

Always Use Filters: Apply the appropriate SQL filter to every parameter
Define Clear Parameters: Use descriptive names and provide sample values
Set Proper Permissions: Restrict query access to appropriate roles
Validate Input Types: Use the built-in type system (string, number, date, boolean, array)
Handle Errors Gracefully: Check Success and provide meaningful error messages
Use Approved Queries: Only execute queries with 'Approved' status
Leverage Categories: Organize queries by functional area or team
Test Parameter Combinations: Verify all conditional blocks work correctly
Document Query Purpose: Add clear descriptions for queries and parameters
Review SQL Security: Regular audit of complex expressions and dynamic SQL

Performance Considerations

Parameter Indexing: Ensure filtered columns have appropriate database indexes
Query Optimization: Use efficient JOINs and WHERE clauses
Result Limiting: Consider adding TOP/LIMIT clauses for large datasets
Caching: Results are not automatically cached - implement application-level caching if needed
Connection Pooling: RunQuery leverages provider connection pooling automatically

Integration with AI Systems

RunQuery is designed to work seamlessly with AI systems:

Token-Efficient Metadata: Filter definitions are optimized for AI prompts
Self-Documenting: Parameter definitions include examples and descriptions
Safe Code Generation: AI can generate queries using the secure filter system
Validation Feedback: Clear error messages help AI systems learn and adapt

Example: Complete Sales Dashboard Query

-- Stored query: "Sales Dashboard Data"
SELECT 
    DATEPART(month, o.OrderDate) AS Month,
    DATEPART(year, o.OrderDate) AS Year,
    COUNT(*) AS OrderCount,
    SUM(o.TotalAmount) AS TotalRevenue,
    AVG(o.TotalAmount) AS AvgOrderValue,
    COUNT(DISTINCT o.CustomerID) AS UniqueCustomers
    {% if includeProductBreakdown %}
    , p.CategoryPath
    , SUM(od.Quantity) AS TotalQuantity
    {% endif %}
FROM Orders o
{% if includeProductBreakdown %}
    INNER JOIN OrderDetails od ON o.ID = od.OrderID
    INNER JOIN Products p ON od.ProductID = p.ID
{% endif %}
WHERE 
    o.OrderDate >= {{ startDate | sqlDate }} AND
    o.OrderDate <= {{ endDate | sqlDate }} AND
    o.Status IN {{ allowedStatuses | sqlIn }}
    {% if filterByRegion %}
    AND o.Region = {{ region | sqlString }}
    {% endif %}
    {% if minOrderValue %}
    AND o.TotalAmount >= {{ minOrderValue | sqlNumber }}
    {% endif %}
GROUP BY 
    DATEPART(month, o.OrderDate),
    DATEPART(year, o.OrderDate)
    {% if includeProductBreakdown %}
    , p.CategoryPath
    {% endif %}
ORDER BY {{ orderExpression | sqlNoKeywordsExpression }}

// Execute the dashboard query
const dashboardResult = await rq.RunQuery({
    QueryName: 'Sales Dashboard Data',
    CategoryPath: '/Analytics/',  // Hierarchical path notation
    Parameters: {
        startDate: '2024-01-01',
        endDate: '2024-12-31',
        allowedStatuses: ['Completed', 'Shipped'],
        includeProductBreakdown: true,
        filterByRegion: true,
        region: 'North America',
        minOrderValue: 100,
        orderExpression: 'Year DESC, Month DESC, TotalRevenue DESC'
    }
});

if (dashboardResult.Success) {
    // Process the comprehensive dashboard data
    const monthlyData = dashboardResult.Results;
    console.log(`Generated dashboard with ${monthlyData.length} data points`);
    console.log(`Parameters applied:`, dashboardResult.AppliedParameters);
}

RunReport Class

Execute reports with various output formats:

import { RunReport, RunReportParams } from '@memberjunction/core';

const rr = new RunReport();

const params: RunReportParams = {
    ReportID: '12345',
    Parameters: {
        Year: 2024,
        Department: 'Sales'
    }
};

const results = await rr.RunReport(params);

Transaction Management

Group multiple operations in a transaction:

import { TransactionGroupBase } from '@memberjunction/core';

// Transaction groups allow you to execute multiple entity operations atomically
// See your specific provider documentation for implementation details

// Example using SQLServerDataProvider:
const transaction = new TransactionGroupBase('MyTransaction');

// Add entities to the transaction
await transaction.AddTransaction(entity1);
await transaction.AddTransaction(entity2);

// Submit all operations as a single transaction
const results = await transaction.Submit();

For instance-level transactions in multi-user environments, each provider instance maintains its own transaction state, providing automatic isolation between concurrent requests.

Entity Relationships

MemberJunction automatically handles entity relationships through the metadata system:

// Load an entity with related data
const order = await md.GetEntityObject<OrderEntity>('Orders');
await order.Load(123, ['OrderDetails', 'Customer']);

// Access related entities
const orderDetails = order.OrderDetails; // Array of OrderDetailEntity
const customer = order.Customer; // CustomerEntity

Security & Permissions

The library provides comprehensive security features:

const md = new Metadata();

// Check current user
const user = md.CurrentUser;
console.log('Current user:', user.Email);

// Check user roles
const isAdmin = user.RoleName.includes('Admin');

// Entity permissions
const entity = md.EntityByName('Orders');
const canCreate = entity.CanCreate;
const canUpdate = entity.CanUpdate;
const canDelete = entity.CanDelete;

Error Handling

All operations return detailed error information:

const entity = await md.GetEntityObject('Users');
const result = await entity.Save();

if (!result) {
    // Access detailed error information
    const error = entity.LatestResult;
    console.error('Error:', error.Message);
    console.error('Details:', error.Details);
    
    // Check validation errors
    if (error.ValidationErrors && error.ValidationErrors.length > 0) {
        error.ValidationErrors.forEach(ve => {
            console.error(`Field ${ve.FieldName}: ${ve.Message}`);
        });
    }
}

Logging

MemberJunction provides enhanced logging capabilities with both simple and advanced APIs:

Basic Logging

import { LogStatus, LogError } from '@memberjunction/core';

// Simple status logging
LogStatus('Operation completed successfully');

// Error logging
LogError('Operation failed', null, additionalData1, additionalData2);

// Logging to file
LogStatus('Writing to file', '/logs/output.log');

Enhanced Logging (v2.59.0+)

The enhanced logging functions provide structured logging with metadata, categories, and conditional verbose output:

LogStatusEx - Enhanced Status Logging

import { LogStatusEx, IsVerboseLoggingEnabled, SetVerboseLogging } from '@memberjunction/core';

// Simple usage - same as LogStatus
LogStatusEx('Process started');

// Verbose-only logging (respects MJ_VERBOSE environment variable)
LogStatusEx({
    message: 'Detailed trace information',
    verboseOnly: true
});

// With custom verbose check
LogStatusEx({
    message: 'Processing items:',
    verboseOnly: true,
    isVerboseEnabled: () => myConfig.debugMode === true,
    additionalArgs: [item1, item2, item3]
});

// With category and file output
LogStatusEx({
    message: 'Batch job completed',
    category: 'BatchProcessor',
    logToFileName: '/logs/batch.log',
    additionalArgs: [processedCount, errorCount]
});

LogErrorEx - Enhanced Error Logging

import { LogErrorEx } from '@memberjunction/core';

// Simple usage - same as LogError
LogErrorEx('Something went wrong');

// With error object and severity
try {
    await riskyOperation();
} catch (error) {
    LogErrorEx({
        message: 'Failed to complete operation',
        error: error as Error,
        severity: 'critical',
        category: 'DataProcessing'
    });
}

// With metadata and additional arguments
LogErrorEx({
    message: 'Validation failed',
    severity: 'warning',
    category: 'Validation',
    metadata: {
        userId: user.ID,
        attemptCount: 3,
        validationRules: ['email', 'uniqueness']
    },
    additionalArgs: [validationResult, user]
});

// Control stack trace inclusion
LogErrorEx({
    message: 'Network timeout',
    error: timeoutError,
    includeStack: false, // Omit stack trace
    metadata: { url: apiUrl, timeout: 5000 }
});

Verbose Logging Control

Control verbose logging globally across your application:

// Check if verbose logging is enabled
if (IsVerboseLoggingEnabled()) {
    // Perform expensive logging operations
    const debugInfo = gatherDetailedDebugInfo();
    LogStatus('Debug info:', debugInfo);
}

// Enable verbose logging in browser environments
SetVerboseLogging(true);

// Verbose logging is controlled by:
// 1. MJ_VERBOSE environment variable (Node.js)
// 2. MJ_VERBOSE global variable (Browser)
// 3. MJ_VERBOSE localStorage item (Browser)
// 4. MJ_VERBOSE URL parameter (Browser)

Logging Features

Severity Levels: warning, error, critical for LogErrorEx
Categories: Organize logs by functional area
Metadata: Attach structured data to logs
Varargs Support: Pass additional arguments that get forwarded to console.log/error
File Logging: Direct logs to files (Node.js environments)
Conditional Logging: Skip verbose logs based on environment settings
Error Objects: Automatic error message and stack trace extraction
Cross-Platform: Works in both Node.js and browser environments

Provider Architecture

MemberJunction uses a provider model to support different execution environments:

import { SetProvider } from '@memberjunction/core';

// Provider setup is typically handled by your application initialization
// The provider determines how data is accessed (direct database, API, etc.)
SetProvider(myProvider);

Metadata Caching Optimization

Starting in v2.0, providers support intelligent metadata caching for improved performance in multi-user environments:

// First provider instance loads metadata from the database
const provider1 = new SQLServerDataProvider(connectionPool);
await provider1.Config(config); // Loads metadata from database

// Subsequent instances can reuse cached metadata
const config2 = new SQLServerProviderConfigData(
  connectionPool, 
  '__mj', 
  0, 
  undefined, 
  undefined, 
  false // ignoreExistingMetadata = false to reuse cached metadata
);
const provider2 = new SQLServerDataProvider(connectionPool);
await provider2.Config(config2); // Reuses metadata from provider1

This optimization is particularly beneficial in server environments where each request gets its own provider instance.

Breaking Changes

v2.59.0

Enhanced Logging Functions: New LogStatusEx and LogErrorEx functions provide structured logging with metadata, categories, and severity levels. The existing LogStatus and LogError functions now internally use the enhanced versions, maintaining full backward compatibility.
Verbose Logging Control: New global functions IsVerboseLoggingEnabled() and SetVerboseLogging() provide centralized verbose logging control across environments.

v2.58.0

GetEntityObject() now calls NewRecord() automatically: When creating new entities, NewRecord() is now called automatically. While calling it again is harmless, it's no longer necessary.
UUID Generation: Entities with non-auto-increment uniqueidentifier primary keys now have UUIDs generated automatically in NewRecord().

v2.52.0

LoadFromData() is now async: The LoadFromData() method in BaseEntity is now async to support subclasses that need to perform additional asynchronous operations during data loading. Update any direct calls to this method to use await.

Best Practices

Always use Metadata.GetEntityObject() to create entity instances - never use new
Use generic types with RunView for type-safe results
Handle errors properly - check return values and LatestResult
Use transactions for related operations that must succeed/fail together
Leverage metadata for dynamic UI generation and validation
Respect permissions - always check CanCreate/Update/Delete before operations
Use ExtraFilter carefully - ensure SQL injection protection
Cache metadata instances when possible to improve performance
Override both Load() and LoadFromData() in subclasses that need additional loading logic to ensure consistent behavior

Integration with Other MemberJunction Packages

@memberjunction/global: Global utilities and constants
@memberjunction/server: Server-side provider implementation
@memberjunction/client: Client-side provider implementation
@memberjunction/angular: Angular-specific components and services
@memberjunction/react: React-specific components and hooks
@memberjunction/ai: AI integration features
@memberjunction/communication: Communication and messaging features

Dependencies

@memberjunction/global: Core global utilities
rxjs: Reactive programming support
zod: Schema validation
debug: Debug logging

TypeScript Support

This library is written in TypeScript and provides full type definitions. All generated entity classes include proper typing for IntelliSense support.

Datasets

Datasets are a powerful performance optimization feature in MemberJunction that allows efficient bulk loading of related entity data. Instead of making multiple individual API calls to load different entities, datasets enable you to load collections of related data in a single operation.

What Are Datasets?

Datasets are pre-defined collections of related entity data that can be loaded together. Each dataset contains multiple "dataset items" where each item represents data from a specific entity. This approach dramatically reduces database round trips and improves application performance.

How Datasets Work

Dataset Definition: Datasets are defined in the Datasets entity with a unique name and description
Dataset Items: Each dataset contains multiple items defined in the Dataset Items entity, where each item specifies:
- The entity to load
- An optional filter to apply
- A unique code to identify the item within the dataset
Bulk Loading: When you request a dataset, all items are loaded in parallel in a single database operation
Caching: Datasets can be cached locally for offline use or improved performance

Key Benefits

Reduced Database Round Trips: Load multiple entities in one operation instead of many
Better Performance: Parallel loading and optimized queries
Caching Support: Built-in local caching with automatic cache invalidation
Offline Capability: Cached datasets enable offline functionality
Consistency: All data in a dataset is loaded at the same point in time

The MJ_Metadata Dataset

The most important dataset in MemberJunction is MJ_Metadata, which loads all system metadata including:

Entities and their fields
Applications and settings
User roles and permissions
Query definitions
Navigation items
And more...

This dataset is used internally by MemberJunction to bootstrap the metadata system efficiently.

Dataset API Methods

The Metadata class provides several methods for working with datasets:

GetDatasetByName()

Always retrieves fresh data from the server without checking cache:

const md = new Metadata();
const dataset = await md.GetDatasetByName('MJ_Metadata');

if (dataset.Success) {
    // Process the dataset results
    for (const item of dataset.Results) {
        console.log(`Loaded ${item.Results.length} records from ${item.EntityName}`);
    }
}

GetAndCacheDatasetByName()

Retrieves and caches the dataset, using cached version if up-to-date:

// This will use cache if available and up-to-date
const dataset = await md.GetAndCacheDatasetByName('ProductCatalog');

// With custom filters for specific items
const filters: DatasetItemFilterType[] = [
    { ItemCode: 'Products', Filter: 'IsActive = 1' },
    { ItemCode: 'Categories', Filter: 'ParentID IS NULL' }
];
const filteredDataset = await md.GetAndCacheDatasetByName('ProductCatalog', filters);

IsDatasetCacheUpToDate()

Checks if the cached version is current without loading the data:

const isUpToDate = await md.IsDatasetCacheUpToDate('ProductCatalog');
if (!isUpToDate) {
    console.log('Cache is stale, refreshing...');
    await md.GetAndCacheDatasetByName('ProductCatalog');
}

ClearDatasetCache()

Removes a dataset from local cache:

// Clear specific dataset
await md.ClearDatasetCache('ProductCatalog');

// Clear dataset with specific filters
await md.ClearDatasetCache('ProductCatalog', filters);

Dataset Filtering

You can apply filters to individual dataset items to load subsets of data:

const filters: DatasetItemFilterType[] = [
    {
        ItemCode: 'Orders',
        Filter: "OrderDate >= '2024-01-01' AND Status = 'Active'"
    },
    {
        ItemCode: 'OrderDetails',
        Filter: "OrderID IN (SELECT ID FROM Orders WHERE OrderDate >= '2024-01-01')"
    }
];

const dataset = await md.GetAndCacheDatasetByName('RecentOrders', filters);

Dataset Caching

Datasets are cached using the provider's local storage implementation:

Browser: IndexedDB or localStorage
Node.js: File system or memory cache
React Native: AsyncStorage

The cache key includes:

Dataset name
Applied filters (if any)
Connection string (to prevent cache conflicts between environments)

Cache Invalidation

The cache is automatically invalidated when:

Any entity in the dataset has newer data on the server
Row counts differ between cache and server
You manually clear the cache

Creating Custom Datasets

To create your own dataset:

Create a record in the Datasets entity:

const datasetEntity = await md.GetEntityObject<DatasetEntity>('Datasets');
datasetEntity.Name = 'CustomerDashboard';
datasetEntity.Description = 'All data needed for customer dashboard';
await datasetEntity.Save();

Add dataset items for each entity to include:

const itemEntity = await md.GetEntityObject<DatasetItemEntity>('Dataset Items');
itemEntity.DatasetID = datasetEntity.ID;
itemEntity.Code = 'Customers';
itemEntity.EntityID = md.EntityByName('Customers').ID;
itemEntity.Sequence = 1;
itemEntity.WhereClause = 'IsActive = 1';
await itemEntity.Save();

Best Practices

Use Datasets for Related Data: When you need multiple entities that are logically related
Cache Strategically: Use GetAndCacheDatasetByName() for data that doesn't change frequently
Apply Filters Wisely: Filters reduce data volume but make cache keys more specific
Monitor Cache Size: Large datasets can consume significant local storage
Refresh When Needed: Use IsDatasetCacheUpToDate() to check before using cached data

Example: Loading a Dashboard

// Define a dataset for a sales dashboard
const dashboardFilters: DatasetItemFilterType[] = [
    { ItemCode: 'Sales', Filter: "Date >= DATEADD(day, -30, GETDATE())" },
    { ItemCode: 'Customers', Filter: "LastOrderDate >= DATEADD(day, -30, GETDATE())" },
    { ItemCode: 'Products', Filter: "StockLevel < ReorderLevel" }
];

// Load with caching for performance
const dashboard = await md.GetAndCacheDatasetByName('SalesDashboard', dashboardFilters);

if (dashboard.Success) {
    // Extract individual entity results
    const recentSales = dashboard.Results.find(r => r.Code === 'Sales')?.Results || [];
    const activeCustomers = dashboard.Results.find(r => r.Code === 'Customers')?.Results || [];
    const lowStockProducts = dashboard.Results.find(r => r.Code === 'Products')?.Results || [];
    
    // Use the data to render your dashboard
    console.log(`Recent sales: ${recentSales.length}`);
    console.log(`Active customers: ${activeCustomers.length}`);
    console.log(`Low stock products: ${lowStockProducts.length}`);
}

License

ISC License - see LICENSE file for details

Vector Embeddings Support (v2.90.0+)

MemberJunction now provides built-in support for generating and managing vector embeddings for text fields in entities. This feature enables similarity search, duplicate detection, and AI-powered features across your data.

Overview

The BaseEntity class now includes methods for generating vector embeddings from text fields and storing them alongside the original data. This functionality is designed to be used by server-side entity subclasses that have access to AI embedding models.

Core Methods

BaseEntity provides four methods for managing vector embeddings:

// Generate embeddings for multiple fields by field names
protected async GenerateEmbeddingsByFieldName(fields: Array<{
    fieldName: string,           // Source text field name
    vectorFieldName: string,     // Target vector storage field name
    modelFieldName: string       // Field to store the model ID used
}>): Promise<boolean>

// Generate embedding for a single field by name
protected async GenerateEmbeddingByFieldName(
    fieldName: string, 
    vectorFieldName: string, 
    modelFieldName: string
): Promise<boolean>

// Generate embeddings for multiple fields using EntityField objects
protected async GenerateEmbeddings(fields: Array<{
    field: EntityField,
    vectorField: EntityField,
    modelField: EntityField
}>): Promise<boolean>

// Generate embedding for a single field
protected async GenerateEmbedding(
    field: EntityField,
    vectorField: EntityField,
    modelField: EntityField
): Promise<boolean>

Implementation Pattern

To use vector embeddings in your entity:

Add Vector Storage Fields to your database table:
- A field to store the vector (typically NVARCHAR(MAX))
- A field to store the model ID that generated the vector
Implement EmbedTextLocal in your server-side entity subclass:

import { BaseEntity, SimpleEmbeddingResult } from "@memberjunction/core";
import { AIEngine } from "@memberjunction/aiengine";

export class MyEntityServer extends MyEntity {
    protected async EmbedTextLocal(textToEmbed: string): Promise<SimpleEmbeddingResult> {
        await AIEngine.Instance.Config(false, this.ContextCurrentUser);
        const result = await AIEngine.Instance.EmbedTextLocal(textToEmbed);
        
        if (!result?.result?.vector || !result?.model?.ID) {
            throw new Error('Failed to generate embedding');
        }
        
        return {
            vector: result.result.vector,
            modelID: result.model.ID
        };
    }
}

Call GenerateEmbeddings in your Save method:

public async Save(): Promise<boolean> {
    // Generate embeddings before saving
    await this.GenerateEmbeddingsByFieldName([
        {
            fieldName: "Description",
            vectorFieldName: "DescriptionVector",
            modelFieldName: "DescriptionVectorModelID"
        },
        {
            fieldName: "Content",
            vectorFieldName: "ContentVector",
            modelFieldName: "ContentVectorModelID"
        }
    ]);
    
    return await super.Save();
}

Automatic Features

The embedding generation system includes several automatic optimizations:

Dirty Detection: Only generates embeddings for new records or when source text changes
Null Handling: Clears vector fields when source text is empty
Parallel Processing: Multiple embeddings are generated concurrently for performance
Error Resilience: Returns false on failure without throwing exceptions

Type Definitions

The SimpleEmbeddingResult type is defined in @memberjunction/core:

export type SimpleEmbeddingResult = {
    vector: number[];    // The embedding vector
    modelID: string;     // ID of the AI model used
}

Architecture Benefits

This design provides:

Clean Separation: Core orchestration logic in BaseEntity, AI integration in subclasses
No Dependency Issues: BaseEntity doesn't depend on AI packages
Reusability: Any server-side entity can add embeddings with minimal code
Type Safety: Full TypeScript support throughout

Example: Complete Implementation

// In your server-side entity file
import { BaseEntity, SimpleEmbeddingResult } from "@memberjunction/core";
import { RegisterClass } from "@memberjunction/global";
import { ComponentEntityExtended } from "@memberjunction/core-entities";
import { AIEngine } from "@memberjunction/aiengine";

@RegisterClass(BaseEntity, 'Components')
export class ComponentEntityServer extends ComponentEntityExtended {
    public async Save(): Promise<boolean> {
        // Generate embeddings for text fields
        await this.GenerateEmbeddingsByFieldName([
            {
                fieldName: "TechnicalDesign",
                vectorFieldName: "TechnicalDesignVector",
                modelFieldName: "TechnicalDesignVectorModelID"
            },
            {
                fieldName: "FunctionalRequirements",
                vectorFieldName: "FunctionalRequirementsVector",
                modelFieldName: "FunctionalRequirementsVectorModelID"
            }
        ]);
        
        return await super.Save();
    }
    
    protected async EmbedTextLocal(textToEmbed: string): Promise<SimpleEmbeddingResult> {
        await AIEngine.Instance.Config(false, this.ContextCurrentUser);
        const e = await AIEngine.Instance.EmbedTextLocal(textToEmbed);
        
        if (!e?.result?.vector || !e?.model?.ID) {
            throw new Error('Failed to generate embedding - no vector or model ID returned');
        }
        
        return {
            vector: e.result.vector,
            modelID: e.model.ID
        };
    }
}

Best Practices

Database Schema: Store vectors as JSON strings in NVARCHAR(MAX) fields
Model Tracking: Always store the model ID to track which model generated each vector
Error Handling: Implement proper error handling in your EmbedTextLocal override
Performance: Use batch methods when generating multiple embeddings
Security: Ensure proper user context is passed for multi-tenant scenarios

Support

For support, documentation, and examples, visit MemberJunction.com