IMPORTANT DISCLAIMER: This guide provides general patterns and approaches for reimplementing Deno support using modern deno_core and deno_runtime crates. The exact API signatures, import paths, and method names may vary by version and should be verified against the actual Deno documentation and source code for the specific versions being used.
Key Points:
- Deno APIs evolve frequently - always verify against current documentation
- Code examples are illustrative patterns, not guaranteed to compile
- Import paths may differ (e.g.,
deno_runtime::permissionsvsdeno_runtime::deno_permissions)- Method signatures should be confirmed in the actual crate documentation
- Testing with the target Deno version is essential
Recommended Approach:
- Start with small working examples using the target Deno version
- Build incrementally, testing each component
- Consult official Deno documentation and examples
- Review Deno source code for internal APIs if needed
This document provides a step-by-step guide for reimplementing Deno support in emacs-ng with the latest Deno versions.
The goal is to restore JavaScript/TypeScript evaluation in emacs-ng using modern Deno crates (deno_core and deno_runtime) while maintaining compatibility with the existing Lisp interface.
Update crates/js/Cargo.toml:
[dependencies]
emacs-sys.path = "../emacs-sys"
lisp-macros.path = "../lisp-macros"
lisp-util.path = "../lisp-util"
lsp-json.path = "../lsp-json"
libc.workspace = true
futures = "0.3"
serde_json = { version = "1.0", features = ["preserve_order"] }
tokio = { workspace = true, features = ["full"] }
# Updated Deno dependencies
deno_core = "0.371"
deno_runtime = "0.229"
url = "2.5"The old code used ProgramState and create_main_worker. The new approach:
// NOTE: These are estimated based on deno_runtime patterns.
// Actual API paths and method signatures should be verified
// against the specific version of deno_runtime being used.
use deno_runtime::permissions::PermissionsContainer;
use deno_runtime::worker::{MainWorker, WorkerOptions};
use deno_core::ModuleSpecifier;
use std::rc::Rc;
fn create_worker(
main_module: ModuleSpecifier,
permissions: PermissionsContainer,
) -> Result<MainWorker, AnyError> {
// Permissions are typically set in WorkerOptions, not passed separately
let options = WorkerOptions {
module_loader: Rc::new(deno_runtime::deno_fs::FsModuleLoader),
// Other required options will need to be configured
..Default::default()
};
// Actual constructor may vary - verify with deno_runtime docs
MainWorker::bootstrap_from_options(main_module, options)
}Old structure had:
tokio_runtimedeno_workerprogram_state- Various state management fields
New structure (simplified):
struct EmacsMainJsRuntime {
tokio_runtime: Option<tokio::runtime::Runtime>,
deno_worker: Option<MainWorker>,
within_runtime: bool,
module_counter: u64,
stacked_v8_handle: Option<*mut v8::HandleScope<'static>>,
options: EmacsJsOptions,
proxy_template: Option<v8::Global<v8::ObjectTemplate>>,
within_toplevel: bool,
tick_scheduled: bool,
}Key changes:
- Remove
program_statefield - Keep core runtime management
- Permissions now part of worker, not separate state
Old:
let permissions = Permissions::from_options(&flags.into());New:
// NOTE: The exact permissions API path may vary by version.
// Common patterns include:
// - deno_runtime::permissions::Permissions
// - deno_runtime::permissions::PermissionsContainer
// Verify the actual API in your version of deno_runtime
// For allow-all (typical pattern):
// let permissions = PermissionsContainer::allow_all();
// For custom permissions, the API typically involves:
// 1. Creating permission options/descriptors
// 2. Building a permissions container
//
// Example pattern (verify against actual API):
// let permissions = PermissionsContainer::new(
// Permissions {
// net: NetPermissions::allow_all(),
// read: ReadPermissions::allow_all(),
// write: WritePermissions::allow_all(),
// run: RunPermissions::allow_all(),
// ..Default::default()
// }
// );Old code used program_state.file_fetcher.insert_cached() to inject modules.
New approach using custom module loader:
use deno_core::{ModuleLoader, ModuleSource, ModuleSourceCode, ModuleType};
use std::pin::Pin;
use futures::future::Future;
struct EmacsModuleLoader {
cached_modules: Arc<Mutex<HashMap<ModuleSpecifier, String>>>,
}
impl ModuleLoader for EmacsModuleLoader {
fn resolve(
&self,
specifier: &str,
referrer: &str,
_kind: deno_core::ResolutionKind,
) -> Result<ModuleSpecifier, AnyError> {
deno_core::resolve_import(specifier, referrer)
}
fn load(
&self,
module_specifier: &ModuleSpecifier,
_maybe_referrer: Option<&ModuleSpecifier>,
_is_dyn_import: bool,
_requested_module_type: deno_core::RequestedModuleType,
) -> Pin<Box<dyn Future<Output = Result<ModuleSource, AnyError>>>> {
let specifier = module_specifier.clone();
let cached = self.cached_modules.clone();
async move {
// Check cache first
if let Some(source) = cached.lock().unwrap().get(&specifier) {
return Ok(ModuleSource::new(
ModuleType::JavaScript,
ModuleSourceCode::String(source.clone().into()),
&specifier,
None,
));
}
// Load from file system
let path = specifier.to_file_path()
.map_err(|_| anyhow::anyhow!("Invalid file path"))?;
let source = tokio::fs::read_to_string(&path).await?;
Ok(ModuleSource::new(
ModuleType::JavaScript,
ModuleSourceCode::String(source.into()),
&specifier,
None,
))
}
.boxed_local()
}
}Old:
worker.execute_module(&main_module).await?;
worker.run_event_loop().await?;New (similar but different context):
// Execute module
let mod_id = worker.preload_main_module(&main_module).await?;
worker.evaluate_module(mod_id).await?;
// Run event loop
worker.run_event_loop(false).await?;For eval_js functionality:
// Old approach used execute_module with cached fake file
// New approach: direct script execution
let result = worker.execute_script(
&format!("./$anon${}.js", counter),
source_code.into(),
)?;The existing code for getting v8 scopes remains similar, but accessing the JsRuntime is different:
Old:
let runtime = &mut worker.js_runtime;
let context = runtime.global_context();
let scope = &mut v8::HandleScope::with_context(runtime.v8_isolate(), context);New:
let js_runtime = &mut worker.js_runtime;
let context = js_runtime.main_context();
let scope = &mut js_runtime.handle_scope();The v8_bind_lisp_funcs function needs minor updates:
pub(crate) fn v8_bind_lisp_funcs(
worker: &mut MainWorker,
) -> Result<(), AnyError> {
let js_runtime = &mut worker.js_runtime;
let scope = &mut js_runtime.handle_scope();
let context = scope.get_current_context();
let global = context.global(scope);
// Rest remains the same: bind functions to global object
bind_global_fn!(scope, global, lisp_invoke);
// ... other bindings
// Execute preliminary JS
js_runtime.execute_script("prelim.js", include_str!("prelim.js"))?;
Ok(())
}Each subcommand needs to be rewritten. Example for eval_command:
pub(crate) async fn eval_command(
code: String,
ext: String,
print: bool,
) -> Result<(), AnyError> {
let main_module = ModuleSpecifier::parse(&format!("file://github.com/./$deno$eval.{}", ext))?;
let permissions = PermissionsContainer::allow_all();
let source_code = if print {
format!("console.log({})", code)
} else {
code
};
// Create module loader with cached code
let module_loader = Rc::new(EmacsModuleLoader::new());
module_loader.insert_cached(main_module.clone(), source_code);
let options = WorkerOptions {
module_loader,
permissions: permissions.clone(),
..Default::default()
};
let mut worker = MainWorker::bootstrap_from_options(
main_module.clone(),
permissions,
options,
)?;
// Bind lisp functions
v8_bind_lisp_funcs(&mut worker)?;
// Execute module
let mod_id = worker.preload_main_module(&main_module).await?;
worker.evaluate_module(mod_id).await?;
worker.run_event_loop(false).await?;
Ok(())
}REPL is more complex and may require using Deno's REPL functionality:
pub(crate) async fn run_repl() -> Result<(), AnyError> {
// This requires deeper integration with Deno's REPL
// May need to use deno_cli crate or implement custom REPL
// For now, this is a placeholder showing the structure
let main_module = ModuleSpecifier::parse("file://github.com/./$deno$repl.ts")?;
let permissions = PermissionsContainer::allow_all();
let options = WorkerOptions {
// REPL-specific options
..Default::default()
};
let mut worker = MainWorker::bootstrap_from_options(
main_module,
permissions,
options,
)?;
v8_bind_lisp_funcs(&mut worker)?;
// REPL loop would go here
// This is complex and may require deno_cli crate
Ok(())
}TypeScript compilation in newer Deno versions is handled differently:
Modern Deno uses deno_ast (which wraps SWC) for TypeScript compilation:
// NOTE: The deno_ast API evolves frequently. This is a general pattern.
// Verify the exact API against your version of deno_ast.
// Add to Cargo.toml:
// deno_ast = "0.42" // Check for latest compatible version
// Example transpilation pattern:
use deno_ast::{MediaType, ParseParams, SourceTextInfo};
fn transpile_typescript(source: &str) -> Result<String, AnyError> {
// The exact API varies by version. Common pattern:
// 1. Parse the source with TypeScript media type
// 2. Transpile to JavaScript
// 3. Extract the transpiled text
// Typical usage (verify against actual deno_ast version):
// let parsed = deno_ast::parse_module(ParseParams {
// specifier: "file://github.com/module.ts".to_string(),
// text_info: SourceTextInfo::from_string(source.to_string()),
// media_type: MediaType::TypeScript,
// capture_tokens: false,
// scope_analysis: false,
// maybe_syntax: None,
// })?;
//
// let transpiled = parsed.transpile(&EmitOptions::default())?;
// Ok(transpiled.text)
// Simplified approach: Let the runtime handle it via module loader
// by setting the correct MediaType
unimplemented!("Check deno_ast documentation for current API")
}For simpler cases, use the built-in transpilation:
// Worker options with TypeScript support
let options = WorkerOptions {
// Enable TypeScript transpilation
// This is handled automatically by the module loader in most cases
..Default::default()
};#[derive(Clone)]
struct EmacsJsOptions {
tick_rate: f64,
permissions: PermissionsContainer, // Changed from Option<Permissions>
error_handler: LispObject,
inspect: Option<String>,
inspect_brk: Option<String>,
use_color: bool,
loops_per_tick: EmacsUint,
}#[lisp_fn]
pub fn js_initialize(args: &[LispObject]) -> LispObject {
let ops = permissions_from_args(args);
EmacsMainJsRuntime::set_options(ops.clone());
js_init_sys(&ops)
.map(|_| emacs_sys::globals::Qt)
.unwrap_or_else(|e| {
error!("JS Failed to initialize with error: {}", e);
})
}
fn js_init_sys(js_options: &EmacsJsOptions) -> Result<(), AnyError> {
init_tokio()?;
init_worker(js_options)?;
Ok(())
}
fn init_worker(js_options: &EmacsJsOptions) -> Result<(), AnyError> {
if EmacsMainJsRuntime::is_main_worker_active() {
return Ok(());
}
let main_module = ModuleSpecifier::parse("file://github.com/init.js")?;
let module_loader = Rc::new(EmacsModuleLoader::new());
let options = WorkerOptions {
module_loader,
permissions: js_options.permissions.clone(),
// Add inspector support if configured
..Default::default()
};
let mut worker = MainWorker::bootstrap_from_options(
main_module,
js_options.permissions.clone(),
options,
)?;
v8_bind_lisp_funcs(&mut worker)?;
EmacsMainJsRuntime::set_deno_worker(worker);
Ok(())
}#[cfg(test)]
mod tests {
use super::*;
#[tokio::test]
async fn test_basic_eval() {
let code = "1 + 1".to_string();
let result = eval_command(code, "js".to_string(), true).await;
assert!(result.is_ok());
}
#[tokio::test]
async fn test_typescript_eval() {
let code = "const x: number = 42; x".to_string();
let result = eval_command(code, "ts".to_string(), false).await;
assert!(result.is_ok());
}
}- Test basic JS evaluation from Lisp
- Test TypeScript compilation
- Test async/await
- Test lisp function calls from JS
- Test JS function calls from Lisp
- Test error handling
- Update Cargo.toml dependencies
- Rewrite EmacsMainJsRuntime structure
- Implement custom ModuleLoader
- Update worker initialization
- Update permissions system
- Rewrite eval_js functions
- Rewrite eval_js_file functions
- Update v8_bind_lisp_funcs
- Rewrite subcommands (eval, run, repl, test)
- Test basic JS evaluation
- Test TypeScript compilation
- Test async operations
- Test lisp-JS interop
- Update documentation
- Performance testing
- TypeScript Compilation: May need
deno_astcrate - REPL: Complex feature, may need
deno_clior custom impl - File Watcher: For
--watchflag, needs different API - Inspector/Debugger: Inspector API may have changed
- Coverage: Code coverage API likely changed
- Module Caching: Need robust caching strategy
- Import Maps: Support may require additional work
- deno_core API docs - Official core API documentation
- deno_runtime API docs - Official runtime API documentation
- Deno source code - Reference implementation
- Deno embedding examples - Official examples
- deno_ast for TypeScript - TypeScript transpilation
- Rusty V8 docs - V8 bindings documentation
When implementing, ensure all versions are compatible:
deno_core,deno_runtime, andrusty_v8must use compatible versions- Check Deno's
Cargo.tomlfor the exact versions they use together - Mismatched versions will cause compilation failures or runtime issues
Before making large changes:
- Create a minimal test project with target Deno versions
- Verify each API call compiles and works
- Test with simple examples before integrating into emacs-ng
- Document any API differences discovered during implementation