AIPL – AI Prompt Language
An enterprise-grade language for AI prompt engineering and agentic workflows
Quick Start • Documentation • Examples • Security • VS Code Extension
📋 Table of Contents
- What is AIPL?
- Key Features
- Quick Start
- LLM Providers
- Security
- Evaluation & Testing
- Monitoring
- Documentation
- Examples
- VS Code Extension
- Installation
- CI/CD
- Contributing
- License
🤔 What is AIPL?
AIPL (AI Prompt Language) is an enterprise-grade programming language for working with Large Language Models (LLMs). It combines simplicity with powerful AI integration, comprehensive security, automated evaluation, and production-ready features for building, testing, and deploying AI applications.
Why AIPL?
| Traditional Approach | AIPL Approach |
|---|---|
| ❌ Scattered prompt strings in code | ✅ Structured, reusable prompt templates |
| ❌ Inconsistent error handling | ✅ Built-in error handling and retries |
| ❌ Manual provider switching | ✅ Easy switching between 15+ LLM providers |
| ❌ Limited workflow orchestration | ✅ Sequential & parallel execution |
| ❌ No IDE support | ✅ Full VS Code integration |
| ❌ Security vulnerabilities | ✅ Built-in sandboxing & validation |
| ❌ No quality metrics | ✅ Comprehensive evaluation framework |
| ❌ Manual deployments | ✅ Automated CI/CD with release notes |
✨ Key Features
🎯 Core Language
- Complete Programming Language – Variables, functions, control flow, error handling
- 100+ Built-in Functions – String manipulation, array operations, file I/O, date/time, HTTP, encoding, validation
- Modern Syntax – Clean, intuitive syntax similar to JavaScript/Python
- Type System ✨ – Static types, interfaces, type inference, generics, runtime validation
- Module System ✨ – Import/export, dependency management, module resolution
- Package Manager (AIPM) ✨ – Install, publish, search packages like npm
- Type Safety – Runtime type checking and validation with strict/lenient modes
- Async/Await – Native support for asynchronous operations
- Advanced Number Formats – Hexadecimal (0xFF), Binary (0b1010) support
- Template Strings – Multiline strings with variable interpolation
🤖 AI Integration
- 14+ LLM Providers – OpenAI, Anthropic, Google, Mistral, Together AI, Groq, Azure OpenAI, Perplexity, Replicate, AI21, Cohere, HuggingFace, Ollama, and more
- Prompt Management – Reusable templates with variable interpolation
- Workflow Orchestration – Build complex multi-step AI workflows with dependencies
- Streaming Support – Real-time response streaming
- Token Management – Track and optimize token usage
- Retry Logic – Automatic retry with exponential backoff
- Rate Limiting – Built-in rate limit management
🛠️ Developer Experience
- VS Code Extension – Full IDE support with syntax highlighting, IntelliSense, debugging
- Multi-Platform – JavaScript, Python, Java, Go SDKs
- CLI Tools – Command-line interface for quick execution
- Server Interfaces – HTTP, WebSocket, gRPC support
- Testing Framework – Built-in testing and validation tools
🚀 Quick Start
Installation
# Using npm npm install -g aipl # Using pip pip install aipl # Using homebrew brew install aipl
Your First AIPL Program
// hello.aipl - Your first AI program
// Call an LLM
let response = llm_call("openai", "What is the meaning of life?", {
model: "gpt-4",
temperature: 0.7
})
// Print the result
print(response.text)
Run it:
aipl hello.aipl
More Complex Example
// Multi-provider AI workflow
// Step 1: Research with GPT-4
let research = llm_call("openai", "Research quantum computing breakthroughs in 2024", {
model: "gpt-4",
maxTokens: 2000
})
// Step 2: Summarize with Claude
let summary = llm_call("anthropic", `Summarize in 3 bullet points: ${research.text}`, {
model: "claude-3-5-sonnet-20241022"
})
// Step 3: Fast fact-check with Groq
let factCheck = llm_call("groq", `Fact-check these claims: ${summary.text}`, {
model: "llama-3.1-70b-versatile"
})
// Output results
print("=== Research Summary ===")
print(summary.text)
print("\n=== Fact Check ===")
print(factCheck.text)
🌐 LLM Providers
AIPL integrates with 10 LLM providers out of the box:
Cloud Providers
| Provider | Models | Best For | Speed | Cost |
|---|---|---|---|---|
| OpenAI | GPT-4, GPT-3.5-turbo | Complex reasoning, general purpose | Medium | $$$ |
| Anthropic | Claude 3 (Opus, Sonnet, Haiku) | Long context, safe responses | Medium | $$$ |
| Gemini 2.0, Gemini Pro | Multimodal, fast responses | Fast | $$ | |
| Mistral AI ✨ | Mistral Large, Medium | European compliance, cost-effective | Fast | $$ |
| Together AI ✨ | Llama 3, Mixtral, CodeLlama | Open-source models | Fast | $ |
| Groq ✨ | Llama 3.1, Mixtral | Ultra-fast inference (100ms) | ⚡ Ultra-Fast | $$ |
| Cohere | Command, Command R+ | Enterprise, embeddings | Medium | $$ |
| HuggingFace | 10,000+ models | Experimentation, variety | Varies | Free/$ |
Local Providers
| Provider | Models | Best For | Speed | Cost |
|---|---|---|---|---|
| Ollama | Llama, Mistral, Phi, etc. | Privacy, offline, development | Fast | Free |
| Llama ✨ | Llama 3.2, 3.1, CodeLlama | Direct local access | Fast | Free |
✨ = New in v1.1
Quick Provider Switching
// Development - use local Ollama
let dev_result = llm_call("ollama", "Test prompt")
// Production - use GPT-4
let prod_result = llm_call("openai", "Test prompt", {model: "gpt-4"})
// Speed-critical - use Groq
let fast_result = llm_call("groq", "Test prompt", {model: "llama-3.1-70b-versatile"})
// Cost-sensitive - use Together AI
let cheap_result = llm_call("together", "Test prompt", {model: "meta-llama/Llama-3-70b-chat-hf"})
🔒 Security
AIPL includes enterprise-grade security features to protect your AI applications:
Security Features
- 🛡️ Input Validation – Automatic sanitization of all inputs (prompts, code, URLs, files, JSON)
- 📦 Sandboxed Execution – Isolated VM environment with no file system or process access
- ⏱️ Resource Limits – Configurable CPU time, memory, and execution limits
- 🚦 Rate Limiting – Built-in rate limiting to prevent abuse (100 req/min default)
- 🔐 Encryption – AES-256 encryption for sensitive data
- 🌐 Domain Control – Whitelist/blacklist for external API calls
- 🔍 Vulnerability Scanning – Automatic detection of dangerous code patterns
- 📊 Security Metrics – Real-time monitoring of security events
Quick Example
const SecurityManager = require('aipl/security');
const security = new SecurityManager({
maxExecutionTime: 30000, // 30 seconds
maxMemory: 512 * 1024 * 1024, // 512MB
rateLimitRequests: 100,
rateLimitWindow: 60000, // 1 minute
allowedDomains: ['api.openai.com', 'api.anthropic.com']
});
// Validate input
const validation = security.validateInput(userPrompt, 'prompt');
if (!validation.valid) {
console.error('Invalid input:', validation.error);
}
// Execute in sandbox
const result = await security.executeSandboxed(code, context, 5000);
📖 Full Documentation: Security Guide
✅ Evaluation & Testing
Comprehensive evaluation framework for assessing AIPL script quality and performance:
Evaluation Metrics
- ⚡ Performance – Execution time, memory usage, LLM call efficiency
- 📝 Quality – Syntax, structure, readability, complexity analysis
- 🤖 LLM – Success rates, latency, token usage
- 🔒 Security – Vulnerability detection, security score
Quick Example
const EvaluationFramework = require('aipl/evaluation');
const evaluator = new EvaluationFramework({
outputDir: './evaluation-results',
passingScore: 70
});
// Evaluate a script
const script = `
agent DataProcessor {
prompt: "Analyze: {input}"
llm: "openai"
temperature: 0.7
}
`;
const result = await evaluator.evaluateScript(script, {
executionTime: 1200,
memoryUsed: 50 * 1024 * 1024,
llmCalls: 1
});
console.log('Score:', result.score); // 0-100
console.log('Passed:', result.passed);
console.log('Metrics:', result.metrics);
Benchmarking
// Run benchmark suite
const benchmarks = {
name: 'Performance Test',
tests: [
{ name: 'Simple Agent', script: '...' },
{ name: 'Complex Workflow', script: '...' }
]
};
const results = await evaluator.runBenchmarks(benchmarks);
console.log('Pass Rate:', results.passRate + '%');
📖 Full Documentation: Evaluation Guide
📊 Monitoring
Production-ready observability with comprehensive logging, metrics, and tracing:
Monitoring Features
- 📝 Structured Logging – JSON logs with multiple levels (error, warn, info, debug, trace)
- 📈 Metrics – Counters, gauges, histograms, timers
- 🔍 Distributed Tracing – Span-based tracing for complex workflows
- ⚡ Performance Monitoring – Execution time, memory usage tracking
- 📊 Prometheus Export – Export metrics in Prometheus format
- 🗄️ Log Retention – Automatic cleanup of old logs
Quick Example
const { getMonitor } = require('aipl/monitoring');
const monitor = getMonitor({
logLevel: 'info',
logToFile: true,
metricsEnabled: true,
tracingEnabled: true
});
// Logging
monitor.info('Processing request', { userId: '123' });
monitor.error('Request failed', error, { requestId: 'abc' });
// Metrics
monitor.incrementCounter('requests.total', 1, { endpoint: '/api' });
monitor.setGauge('active.users', 42);
monitor.recordHistogram('response.time', 150);
// Performance timing
const timerId = monitor.startTimer('llm.call', { provider: 'openai' });
const response = await callLLM();
const duration = monitor.endTimer(timerId);
// Tracing
const span = monitor.startSpan('process.workflow');
// ... do work ...
monitor.endSpan(span);
// Get metrics
const metrics = monitor.getAllMetrics();
const prometheus = monitor.exportPrometheus();
📖 Full Documentation: Monitoring Guide
📚 Documentation
Getting Started
Language Reference
Production Features
- 🔒 Security Guide – Sandboxing, validation, encryption
- ✅ Evaluation Guide – Testing, benchmarking, metrics
- 📊 Monitoring Guide – Logging, metrics, tracing
Advanced Topics
Development
💡 Examples
Basic LLM Call
let answer = llm_call("openai", "Explain quantum entanglement simply")
print(answer.text)
With Options
let creative = llm_call("mistral", "Write a haiku about coding", {
model: "mistral-large-latest",
temperature: 0.9,
maxTokens: 100
})
print(creative.text)
Multi-Step Workflow
// Define a function
function analyzeText(text) {
let sentiment = llm_call("openai", `Analyze sentiment: ${text}`)
let summary = llm_call("anthropic", `Summarize: ${text}`)
return {
sentiment: sentiment.text,
summary: summary.text
}
}
// Use it
let article = readFile("article.txt")
let analysis = analyzeText(article)
print(analysis)
Error Handling
try {
let result = llm_call("openai", "Complex query", {
timeout: 30000
})
print(result.text)
} catch (error) {
print("Error:", error.message)
// Fallback to different provider
let fallback = llm_call("groq", "Complex query")
print(fallback.text)
}
Parallel Execution
// Process multiple prompts in parallel
let prompts = [
"Explain AI",
"Explain ML",
"Explain Deep Learning"
]
let results = map(prompts, (prompt) => {
return llm_call("groq", prompt, {model: "llama-3.1-8b-instant"})
})
forEach(results, (result) => {
print(result.text)
print("---")
})
More examples in docs/examples
🔄 CI/CD
AIPL includes production-ready CI/CD pipeline with automated testing, security scanning, and release management:
Features
- ✅ Automated Testing – Unit, integration, and evaluation tests
- 🔒 Security Scanning – npm audit, Snyk vulnerability scanning
- 📊 Code Quality – ESLint, Prettier, complexity analysis
- 🚀 Automated Releases – Semantic versioning with auto-generated release notes
- 📝 Release Notes – AI-powered changelog generation from commits
- 🏗️ Multi-platform – Test on Node 16.x, 18.x, 20.x
- 📈 Code Coverage – Automatic coverage reporting to Codecov
- 🎯 Performance Benchmarks – Automated performance testing
GitHub Actions Workflow
The pipeline automatically:
- Lints code on every push
- Scans for security vulnerabilities
- Tests on multiple Node versions
- Evaluates AIPL scripts for quality
- Builds documentation
- Runs performance benchmarks
- Versions based on commit messages
- Releases with auto-generated notes
- Publishes to NPM
Commit Convention
Use conventional commits for automatic versioning:
# Breaking changes (major version bump) git commit -m "feat!: redesign API structure" git commit -m "feat: add feature BREAKING CHANGE: removes old API" # New features (minor version bump) git commit -m "feat: add support for AWS Bedrock" git commit -m "feat: implement distributed tracing" # Bug fixes (patch version bump) git commit -m "fix: resolve memory leak in parser" git commit -m "fix: correct token counting"
Automated Release Notes
Release notes are automatically generated from commits:
## Release v1.2.0 **Release Date:** 2024-12-06 ### 🚨 Breaking Changes - Redesign API structure for better consistency ### ✨ New Features - Add support for AWS Bedrock models - Implement distributed tracing - Add comprehensive security sandbox ### 🐛 Bug Fixes - Resolve memory leak in parser - Correct token counting for Claude models - Fix rate limiting edge cases ### 📊 Metrics - Test Coverage: 94% - Security Score: A+ - Performance: +15% faster
Pipeline Configuration
View the full pipeline: .github/workflows/ci-cd.yml
Running Locally
# Run tests npm test # Run security audit npm audit # Run evaluation npm run evaluate # Build documentation npm run docs:build # Run all checks npm run prepublishOnly
🎨 VS Code Extension
Get the full IDE experience with the official AIPL extension:
Features
- ✨ Syntax Highlighting – Full language support
- 🔍 IntelliSense – Auto-completion for all 10 LLM providers
- 📝 Code Snippets – Quick scaffolding for common patterns
- 🐛 Debugging – Step-through debugging with breakpoints
- ▶️ Run & Test – Execute AIPL files directly from VS Code
- 🔄 Live Preview – See results in real-time
- 🎯 Provider Switching – Easily switch between LLM providers
Installation
- Open VS Code
- Search for “AIPL” in Extensions
- Click Install
Or install from command line:
code --install-extension aipl.aipl-vscode
Quick Start with VS Code
- Create a new file:
hello.aipl - Type
llmand press Tab for a snippet - Press
Cmd/Ctrl + Shift + Rto run - See results in the output panel
📦 Installation
Prerequisites
- Node.js 16+ (for JavaScript SDK)
- Python 3.8+ (for Python SDK)
- Java 11+ (for Java SDK)
- Go 1.19+ (for Go SDK)
Package Managers
# NPM npm install -g aipl # Yarn yarn global add aipl # PNPM pnpm add -g aipl # Pip pip install aipl # Homebrew (macOS) brew install aipl # Chocolatey (Windows) choco install aipl
From Source
git clone https://github.com/isathish/aipl.git cd aipl npm install npm run build npm link
Verify Installation
aipl --version # Output: 1.0.0 aipl --help # Shows available commands
🔑 API Keys Setup
AIPL needs API keys to access LLM providers. Set them as environment variables:
Quick Setup
# Create .env file cat > .env << EOF # OpenAI OPENAI_API_KEY=sk-... # Anthropic ANTHROPIC_API_KEY=sk-ant-... # Google GOOGLE_API_KEY=AI... # Mistral AI MISTRAL_API_KEY=... # Together AI TOGETHER_API_KEY=... # Groq GROQ_API_KEY=gsk_... # Cohere COHERE_API_KEY=... # HuggingFace HUGGINGFACE_API_KEY=hf_... EOF
Get API Keys
- OpenAI: https://platform.openai.com/api-keys
- Anthropic: https://console.anthropic.com/
- Google: https://makersuite.google.com/app/apikey
- Mistral: https://console.mistral.ai/
- Together AI: https://api.together.xyz/signup
- Groq: https://console.groq.com/
- Cohere: https://dashboard.cohere.com/api-keys
- HuggingFace: https://huggingface.co/settings/tokens
- Ollama: No API key needed (local)
🧪 Testing
AIPL includes a comprehensive testing framework:
// test_example.aipl
// Test basic LLM call
test("OpenAI responds correctly", () => {
let result = llm_call("openai", "Say hello")
assert(result.text.length > 0)
assert(result.model == "gpt-4")
})
// Test with multiple providers
test("All providers work", () => {
let providers = ["openai", "anthropic", "groq"]
forEach(providers, (provider) => {
let result = llm_call(provider, "Test")
assert(result.text != null)
})
})
Run tests:
aipl test test_example.aipl
🏗️ Project Structure
aipl/
├── src/ # Core interpreter
│ ├── lexer.js # Tokenizer
│ ├── parser.js # AST parser
│ ├── interpreter.js # Execution engine
│ ├── llm.js # LLM provider integrations
│ └── builtins.js # Built-in functions
├── aipl/ # Additional resources
│ ├── docs/ # Comprehensive documentation
│ └── sdk/ # SDK implementations
├── vscode-aipl/ # VS Code extension
├── cli/ # Command-line interface
├── server/ # Server implementations
├── examples/ # Example AIPL programs
└── docs/ # Project documentation
🤝 Contributing
We welcome contributions! Here’s how to get started:
- Fork the repository
- Create a feature branch:
git checkout -b feature/amazing-feature - Make your changes and add tests
- Commit:
git commit -m 'Add amazing feature' - Push:
git push origin feature/amazing-feature - Open a Pull Request
Development Setup
# Clone your fork git clone https://github.com/YOUR_USERNAME/aipl.git cd aipl # Install dependencies npm install # Run tests npm test # Build npm run build # Link for local development npm link
Guidelines
- Follow existing code style
- Add tests for new features
- Update documentation
- Keep commits atomic and well-described
🤖 Multi-Agent Orchestration
AIPL v1.2 introduces powerful multi-agent orchestration capabilities for building complex AI workflows.
Agent Types
- Coordinator – Manages workflow execution and task distribution
- Worker – Executes assigned tasks
- Supervisor – Monitors and validates agent performance
- Specialist – Handles domain-specific tasks
- Validator – Verifies output quality
Features
const { AgentOrchestrator, Agent } = require('aipl/multi-agent');
// Create orchestrator
const orchestrator = new AgentOrchestrator({
enableLoadBalancing: true,
enableFailover: true,
healthCheckInterval: 10000
});
// Register agents
const planner = new Agent('planner-1', 'coordinator', ['planning']);
const executor = new Agent('executor-1', 'worker', ['execution']);
orchestrator.registerAgent(planner);
orchestrator.registerAgent(executor);
// Define workflow with dependencies
const workflow = {
tasks: [
{ id: 'analyze', capability: 'planning', handler: analyzeTask },
{ id: 'execute', capability: 'execution', handler: executeTask }
],
dependencies: { execute: ['analyze'] }
};
// Execute workflow
orchestrator.start();
const result = await orchestrator.executeWorkflow(workflow);
Key Capabilities:
- Task dependency graphs
- Parallel execution
- Load balancing
- Automatic failover
- Message routing
- Shared state management
- Health monitoring
⛓️ Advanced Prompt Chaining
Build sophisticated multi-step prompts with context passing and conditional routing.
Chain Types
const { ChainBuilder } = require('aipl/prompt-chain');
// Create a complex chain
const chain = new ChainBuilder('analysis')
// Step 1: Initial prompt
.prompt('summarize', 'Summarize: {{input}}')
// Step 2: Transform result
.transform('extract', (text) => ({
wordCount: text.split(' ').length,
sentences: text.split('.').length
}))
// Step 3: Conditional branching
.if('check-length', 'variables.wordCount > 100',
[{ type: 'prompt', template: 'Provide detailed analysis' }],
[{ type: 'prompt', template: 'Provide brief analysis' }]
)
// Step 4: Parallel processing
.parallel('multi-analysis', [
{ type: 'prompt', template: 'Analyze sentiment: {{summarize}}' },
{ type: 'prompt', template: 'Extract keywords: {{summarize}}' }
])
// Step 5: Aggregate results
.aggregate('combine', 'object', ['check-length', 'multi-analysis'])
// Set variables and execute
.withVariables({ input: 'Your text here' })
.build();
const result = await chain.execute();
Step Types:
prompt– LLM prompt executiontransform– Data transformationconditional– If/else branchingaggregate– Combine multiple resultsparallel– Execute steps simultaneouslyloop– Iterate over databranch– Multi-way conditional
🔍 Vector Database Integration
Native support for multiple vector databases enabling RAG (Retrieval-Augmented Generation).
Supported Databases
- Pinecone – Cloud-native vector database
- Weaviate – GraphQL-based vector search
- Qdrant – High-performance vector database
- Chroma – Lightweight embedding database
- Memory – In-memory for testing
RAG Implementation
const { VectorDBFactory, RAGHelper } = require('aipl/vector-db');
// Connect to vector database
const vectorDB = VectorDBFactory.createClient('pinecone', {
apiKey: process.env.PINECONE_API_KEY,
environment: 'us-west1-gcp',
indexName: 'my-docs'
});
await vectorDB.connect();
// Create RAG helper
const rag = new RAGHelper(vectorDB, embeddingFunction);
// Index documents
await rag.indexDocuments('knowledge-base', [
{
id: 'doc1',
text: 'AIPL is a powerful prompt engineering language',
metadata: { category: 'docs', version: '1.2' }
},
{
id: 'doc2',
text: 'Multi-agent orchestration enables complex workflows',
metadata: { category: 'features', version: '1.2' }
}
]);
// Generate with context retrieval
const response = await rag.generateWithContext(
'knowledge-base',
'What is AIPL?',
llmFunction,
{
contextLimit: 5,
promptTemplate: 'Context: {{context}}\n\nQuestion: {{query}}\n\nAnswer:'
}
);
console.log(response.answer);
console.log('Context used:', response.context);
Features:
- Unified API across databases
- Semantic search
- Metadata filtering
- Batch operations
- Cosine/Euclidean/Dot product similarity
🤝 Real-Time Collaboration
Enable multiple users to collaborate on AIPL scripts in real-time.
Collaboration Server
const { CollaborationServer } = require('aipl/collaboration');
// Start collaboration server
const server = new CollaborationServer({
port: 8080,
maxUsersPerDocument: 50,
enableAuth: true,
authHandler: async (credentials) => ({
success: true,
userId: credentials.userId,
userName: credentials.userName
})
});
server.start();
// Listen for events
server.on('started', () => {
console.log('Collaboration server running on port 8080');
});
server.on('conversion', (data) => {
console.log('User action:', data);
});
Client Usage
const ws = new WebSocket('ws://localhost:8080');
// Join document
ws.send(JSON.stringify({
type: 'join',
payload: {
documentId: 'script-123',
userId: 'user-456',
userName: 'John Doe'
}
}));
// Send edit
ws.send(JSON.stringify({
type: 'edit',
payload: {
type: 'insert',
position: 10,
content: 'new code'
}
}));
// Update cursor position
ws.send(JSON.stringify({
type: 'cursor',
payload: { line: 5, column: 10 }
}));
Features:
- Operational Transformation (OT) for conflict resolution
- Presence awareness (cursors, selections)
- Document locking
- Version history
- Real-time chat
- User color coding
☁️ Cloud Deployment
Deploy AIPL applications to AWS, Azure, GCP, or Kubernetes with a single command.
AWS Lambda Deployment
const { CloudDeploymentFactory } = require('aipl/cloud-deploy');
const deployer = CloudDeploymentFactory.createDeployer('aws', {
projectName: 'my-aipl-app',
region: 'us-east-1',
strategy: 'serverless',
env: {
OPENAI_API_KEY: process.env.OPENAI_API_KEY
}
});
const result = await deployer.deploy();
console.log('Deployed to:', result.url);
Azure Functions
const deployer = CloudDeploymentFactory.createDeployer('azure', {
projectName: 'my-aipl-app',
resourceGroup: 'aipl-rg',
location: 'eastus',
strategy: 'serverless'
});
await deployer.deploy();
Kubernetes
const deployer = CloudDeploymentFactory.createDeployer('kubernetes', {
projectName: 'my-aipl-app',
namespace: 'production',
replicas: 3,
containerImage: 'myregistry/aipl-app:latest'
});
await deployer.deploy();
Deployment Strategies:
- Serverless (Lambda, Functions, Cloud Functions)
- Container (ECS, ACI, Cloud Run)
- VM (EC2, Azure VMs, Compute Engine)
- Kubernetes (any cluster)
Features:
- Terraform infrastructure-as-code
- Automated configuration
- Resource management
- Status monitoring
- Easy teardown
🧪 A/B Testing Framework
Test different prompts and configurations to optimize AI performance.
Create Experiments
const { ABTestingManager } = require('aipl/ab-testing');
const manager = new ABTestingManager();
// Create experiment
const experiment = manager.createExperiment('prompt-optimization', {
allocationStrategy: 'weighted',
trafficPercentage: 50,
targetMetric: 'conversion_rate'
});
// Add variants
experiment.addVariant({
id: 'control',
name: 'Original Prompt',
weight: 1,
prompt: 'Generate a summary of: {{text}}',
parameters: { temperature: 0.7 }
});
experiment.addVariant({
id: 'variant-a',
name: 'Improved Prompt',
weight: 2,
prompt: 'Create a concise summary focusing on key points: {{text}}',
parameters: { temperature: 0.5 }
});
experiment.addVariant({
id: 'variant-b',
name: 'Detailed Prompt',
weight: 1,
prompt: 'Provide a comprehensive summary with examples: {{text}}',
parameters: { temperature: 0.9 }
});
// Start experiment
experiment.start();
Allocate and Track
// Allocate variant to user
const variant = experiment.allocateVariant('user-123');
console.log('Using variant:', variant.name);
// Execute with variant prompt
const result = await executeLLM(variant.prompt, variant.parameters);
// Record conversion (e.g., user satisfaction)
if (userSatisfied) {
experiment.recordConversion('user-123');
}
// Get results with statistical analysis
const results = experiment.getResults();
console.log('Winner:', results.winner);
console.log('Statistics:', results.statistics);
Features:
- Multiple allocation strategies (random, weighted, user-based, sequential)
- Statistical significance testing (z-score, p-value)
- Confidence intervals
- Traffic percentage control
- Custom metrics
- Real-time analytics
🏢 Enterprise Features
Production-ready security, compliance, and access control.
Role-Based Access Control (RBAC)
const { EnterpriseManager, Permission } = require('aipl/enterprise');
const enterprise = new EnterpriseManager();
// Assign roles
enterprise.rbac.assignRole('user-123', 'developer');
enterprise.rbac.assignRole('user-456', 'operator');
enterprise.rbac.assignRole('user-789', 'auditor');
// Check permissions
if (enterprise.checkPermission('user-123', Permission.DEPLOY_SCRIPT)) {
await deployScript();
}
// Authorize with audit logging
await enterprise.authorize(
'user-123',
Permission.EXECUTE_SCRIPT,
{ type: 'script', id: 'production-workflow' },
{ ip: '192.168.1.100', sessionId: 'sess-abc' }
);
Built-in Roles:
- Admin – Full access
- Developer – Development and deployment
- Operator – Operations and monitoring
- Viewer – Read-only access
- Auditor – Audit and compliance
Permissions:
- Resource management (create, read, update, delete)
- User management
- Script execution and deployment
- Configuration changes
- Secrets management
- Audit log access
Single Sign-On (SSO)
const enterprise = new EnterpriseManager({
ssoEnabled: true,
sso: {
provider: 'saml', // or 'oauth2', 'oidc'
issuer: 'https://idp.example.com',
audience: 'aipl-production',
certificate: fs.readFileSync('cert.pem'),
callbackUrl: 'https://aipl.example.com/auth/callback'
}
});
// Initiate SSO authentication
const authRequest = await enterprise.authenticateSSO(request);
// Redirect user to authRequest.redirectUrl
// Validate SSO response
const user = await enterprise.validateSSOResponse(response);
console.log('Authenticated user:', user.email);
Supported Providers:
- SAML 2.0
- OAuth 2.0
- OpenID Connect
Audit Logging
// Automatic audit logging for all actions
await enterprise.audit.log(
'script_executed',
{
userId: 'user-123',
username: 'john.doe',
ip: '192.168.1.100',
userAgent: 'Mozilla/5.0...'
},
{
type: 'script',
id: 'workflow-456',
name: 'production-pipeline'
},
{
duration: 5234,
status: 'success'
}
);
// Query audit log
const events = enterprise.audit.query({
eventType: 'script_executed',
userId: 'user-123',
startDate: Date.now() - 30 * 24 * 60 * 60 * 1000, // Last 30 days
limit: 100
});
// Export audit log
const csvData = enterprise.audit.export('csv');
await fs.writeFile('audit-log.csv', csvData);
Compliance Reporting
// Generate compliance report
const report = await enterprise.getComplianceReport(
startDate,
endDate
);
console.log('Total events:', report.summary.totalEvents);
console.log('Unique users:', report.summary.uniqueActors);
console.log('Unauthorized attempts:', report.summary.unauthorizedAttempts);
// Compliance checks
console.log('GDPR compliant:', report.compliance.gdpr.compliant);
console.log('SOX compliant:', report.compliance.sox.compliant);
console.log('HIPAA compliant:', report.compliance.hipaa.compliant);
Compliance Standards:
- GDPR (data access logging)
- SOX (segregation of duties)
- HIPAA (access control and audit)
📊 Roadmap
v1.1 (Completed ✅ – Q4 2024)
- ✅ Add 4 new LLM providers (Mistral, Together, Groq, Llama)
- ✅ Enhanced VS Code extension with rich UI
- ✅ Comprehensive documentation
- ✅ Type system with interfaces and generics
- ✅ Module system with imports/exports
- ✅ Package manager (AIPM) with CLI
- ✅ Security module with sandboxing
- ✅ Evaluation framework with metrics
- ✅ Monitoring system with observability
- ✅ CI/CD pipeline with automated releases
v1.2 (In Progress 🚧 – Q1 2025)
- 🚧 Multi-agent orchestration system
- 🚧 Advanced prompt chaining with context
- 🚧 Vector database integration (Pinecone, Weaviate, Qdrant, Chroma)
- 🚧 Real-time collaboration features
- 🚧 Cloud deployment tools (AWS, Azure, GCP)
v2.0 (Planned 📋 – Q2 2025)
- 📋 Browser-based playground with Monaco editor
- 📋 Prompt versioning & A/B testing framework
- 📋 Enterprise features (SSO, RBAC, audit logs)
- 📋 Mobile SDK support (iOS & Android)
- 📋 AI model fine-tuning integration
📄 License
MIT License – see LICENSE file for details