Optimizable Features: Barrel Files

Reference Materials

Speeding up the JavaScript ecosystem - The barrel file debacle

How we optimized package imports in Next.js

eslint-plugin-barrel-files

Definition Of Barrel Files

A barrel file is a file (typically named index) that only re-exports references from other files, used to consolidate multiple related feature modules into a unified interface. Essentially, it's similar to npm, encapsulating internal structure and providing a unified interface for consuming modules.

acorn

{
  "name": "acorn",
  "exports": {
    ".": [
      {
        "import": "./dist/acorn.mjs",
        "require": "./dist/acorn.js",
        "default": "./dist/acorn.js"
      },
      "./dist/acorn.js"
    ],
    "./package.json": "./package.json"
  }
}

Barrel files that only adjust directory structure, reduce module import paths, and optimize directory structure features are acceptable. However, in actual projects, the use of barrel files is not standardized. The dependencies within are not mutually independent, but rather have inter-module dependencies (side effects, execution order, etc.) and numerous side effect issues related to modules.

We cannot deny the convenience brought by barrel files, but at the same time, I believe that bundlers and their downstream runtime-related tools (such as vite, test runner, browser, node runtime, etc.) need to make some optimizations in module resolution for barrel files, otherwise it will bring significant performance overhead.

Impact Of Bundlers

After generating a complete module dependency graph in the build phase, bundlers can optimize barrel files by enabling tree-shaking, including only the actually dependent modules in the final output. However, the problem is that bundlers cannot definitively know which modules are actually needed before building the complete module dependency graph.

Bundlers need to recursively resolve every module, including barrel files. Only after resolving all modules in the dependency graph will they enter the tree-shaking phase, where unused modules (sideEffects = 'false') will be cleaned up.

This means that even if most of the dependent modules in barrel files are not actually used, before the complete module dependency graph is resolved (i.e., before the tree-shaking phase), bundlers will equally resolve modules that will eventually be tree-shaken and modules that are actually needed.

From a god's perspective, we don't actually want bundlers to resolve every module that barrel files depend on, but only need to resolve the modules that are actually referenced. However, for bundlers, before the complete module dependency graph is resolved, they cannot obtain sufficient context information to determine which modules need to be used during the resolution process, so they can only treat all modules equally.

This leads to an optimization problem related to barrel files. If barrel files modules are very bloated, containing numerous re-exported references from other dependent modules and widely used in the project, it will inevitably consume a lot of resolution time for bundlers.

Impact Of Runtime

Tree-shaking is a feature of bundling tools, not a javascript runtime feature. For tools that depend on runtime, bloated and numerous barrel files modules will bring significant performance overhead during the execution phase.

Taking vite as an example, the development phase relies on browser support for esm to achieve instant compilation and rendering. However, the drawback is that during the execution phase, vite does not know the complete module dependency graph, and modules don't have enough context to perform tree-shaking optimization. Therefore, when barrel files contain complex side effects, vite appears powerless compared to bundlers.

Currently, when vite processes barrel files modules, the browser will concurrently request the server side. In large-scale barrel files scenarios, numerous concurrent requests have uncontrollable factors for the browser. Even with http 2 multiplexing, a large number of concurrent requests will significantly increase loading time. If using http requests in the development phase, even with extremely low local network latency and no network bandwidth limitations, it will still bring numerous request waterfalls, challenging the browser's limits and causing request stagnation issues. This is more severe compared to unified processing on the node side by bundlers. Therefore, for tools that depend on runtime, special optimization for barrel files is even more necessary.

Vite excels at quick startup. If it spends a lot of performance on semantic analysis of all modules in barrel files, it would be putting the cart before the horse, significantly increasing vite's startup time. In most cases, barrel files don't contain complex side effects in actual production, so vite can provide users with a configuration option to optimize barrel files, but users need to ensure that the dependencies within the optimized barrel files modules are mutually independent.

Optimize Barrel Files

The barrel files feature is also widely used in the community. I believe that for internal module re-exports, if we only need tools to provide path rewriting work, then we can optimize this type of barrel files usage scenario.

Bundlers can provide an option that allows users to configure barrel files that need optimization, with the premise that users need to ensure there are no side effects in the barrel files modules.

In other words, bundlers can help users optimize pure barrel files.

Optimization Achieved

Next.js Optimization

The article How we optimized package imports in Next.js details how next.js optimized barrel files and achieved significant performance improvements.

To improve speed, next.js internally designed a specialized swc transform for optimizeBarrelExports on the rust side. This transformer only focuses on generating the export map and doesn't parse other cases.

Rust Optimization

Prospect Informs

1. When an importer uses wildcard export to depend on a module, the dependent module enters wildcard mode.

2. name_str is the reference name in the current module context, and orig_str is the reference name in the dependent module context.

   • Direct Export

     export { foo } from './foo';

     Both name_str and orig_str are foo.

   • Renamed Export

     export { foo as bar } from './foo';

     name_str is bar, and orig_str is foo.

   • Wildcard Export

     export * as utils from './utils';

     name_str is utils, and orig_str is *.

3. export_map

   export_map is an array of tuples that records export information from export statements.

   // Exported meta information.
   let mut export_map: Vec<(String, String, String)> = vec![];

   Each tuple contains three elements: (name_str, src, orig_str)

   Where src records the source file path if it's re-exported from another file. If it's directly exported in the module, it's an empty string.

   • Namespace Export (ExportSpecifier::Namespace):

     // Code
     export * as utils from './utils';
     
     // export_map record
     export_map.push(
       ('utils', // name_str: exported name
       './utils', // src: source file path
       '*') // orig_str: uses "*" to indicate entire namespace
     );

   • Named Export (ExportSpecifier::Named):

     // Case 1: Re-export source file reference.
     export { foo } from './foo';
     export_map.push(
       ('foo', // name_str: exported name
       './foo', // src: source file path
       'foo') // orig_str: original name
     );
     
     // Case 2: Re-export source file reference with renaming.
     export { foo as bar } from './foo';
     export_map.push(
       ('bar', // name_str: new export name
       './foo', // src: source file path
       'foo') // orig_str: original name
     );
     
     // Case 3: Export module's import declaration identifier.
     import { foo } from './source.js';
     export { foo };
     export_map.push(
       ('foo', // name_str: export name
       './source.js', // src: original source file path
       'foo') // orig_str: original name
     );

   • Export Declaration (ExportDecl):

     // Case 1: Function declaration
     export function hello() {}
     // export_map record
     'hello', // name_str: function name
       '', // src: empty string, because it's a local declaration
       ''; // orig_str: empty string, because it's a local declaration
     
     // Case 2: Class declaration
     export class MyClass {}
     // export_map record
     'MyClass', // name_str: class name
       '', // src: empty string
       ''; // orig_str: empty string
     
     // Case 3: Variable declaration
     export const x = 42;
     // export_map record
     'x', // name_str: variable name
       '', // src: empty string
       ''; // orig_str: empty string

name_str:

• The variable identifier declared in the module context, the name used when finally exporting

src:

• If re-exported from another module, then it's the source file path
• If it's a module declaration, then it's an empty string

orig_str:

• If it's a named export, then it's the original name
• If it's a namespace export, then it's *
• If it's a module declaration, then it's an empty string

This export_map design allows:

• Tracking the source of each export
• Knowing if exports are renamed
• Distinguishing between local declaration exports and re-exports
• Identifying namespace exports

Namespace Export

First, handle the case of ExportSpecifier::Namespace:

Define name_str as the reference name in the current module context for exporting.

ExportSpecifier::Namespace(s) => {
  let name_str = match &s.name {
    ModuleExportName::Ident(n) => n.sym.to_string(),
    ModuleExportName::Str(n) => n.value.to_string(),
  };
}

Discuss three cases:

if let Some(src) = &export_named.src {
  // case1: there is a source file
  export_map.push((name_str.clone(), src.value.to_string(), "*".to_string()));
} else if self.wildcard {
  // case2: no source file and module is in wildcard mode
  export_map.push((name_str.clone(), "".into(), "*".to_string()));
} else {
  // case3: no source file and module is in non-wildcard mode
  is_barrel = false;
  break;
}

And record the export mapping through export_map.

// case1: export statement has source file
// conform to `barrel files` specification
export * as utils from './utils';

// case2: export statement has no source file
import * as utils from './utils';
// module in wildcard mode determines conform to `barrel files` specification.
export { utils };

// case3: export statement has no source file
import * as utils from './utils';
// module in non-wildcard mode determines not conform to `barrel files` specification.
export { utils };

Named Export

Define the reference name for exporting, divided into orig_str and name_str.

let orig_str = match &s.orig {
  ModuleExportName::Ident(n) => n.sym.to_string(),
  ModuleExportName::Str(n) => n.value.to_string(),
};
let name_str = match &s.exported {
  Some(n) => match &n {
    ModuleExportName::Ident(n) => n.sym.to_string(),
    ModuleExportName::Str(n) => n.value.to_string(),
  },
  None => orig_str.clone(),
};

Where orig_str is the original reference name in the export statement, and name_str is the new reference name in the export statement.

Collect different information to export_map:

if let Some(src) = &export_named.src {
  // case1: re-export other module's reference.
  export_map.push((name_str.clone(), src.value.to_string(), orig_str.clone()));
} else if let Some((src, orig)) = local_idents.get(&orig_str) {
  // case2: directly export reference from self module declaration.
  export_map.push((name_str.clone(), src.clone(), orig.clone()));
} else if self.wildcard {
  // case3: wildcard mode allows other export methods.
  export_map.push((name_str.clone(), "".into(), orig_str.clone()));
} else {
  // case4: non-wildcard mode does not allow other export methods.
  is_barrel = false;
  break;
}

// case1: re-export other module's reference.
export { foo } from './foo';
export { bar as baz } from './bar';

// case2: directly export reference from self module.
import { foo } from './foo';
export { foo };

// case3: wildcard mode's other export
const x = 42;
export { x }; // only allowed in wildcard mode

// case4: non-wildcard mode's other export
const x = 42;
export { x }; // in non-wildcard mode not allowed

Wildcard Export

ModuleDecl::ExportAll(export_all) => {
  export_wildcards.push(export_all.src.value.to_string());
}

This handles the * wildcard export case:

export * from './foo';

This is always allowed because it's a typical barrel files behavior.

The complete implementation logic is as follows:

optimize_barrel.rs

// code path: next.js/crates/next-custom-transforms/src/transforms/optimize_barrel.rs
let mut is_barrel = true;
for item in &items {
  match item {
    ModuleItem::ModuleDecl(decl) => {
      allowed_directives = false;
      match decl {
        ModuleDecl::Import(_) => {}
        // export { foo } from './foo';
        ModuleDecl::ExportNamed(export_named) => {
            for spec in &export_named.specifiers {
              match spec {
                ExportSpecifier::Namespace(s) => {
                  let name_str = match &s.name {
                    ModuleExportName::Ident(n) => n.sym.to_string(),
                    ModuleExportName::Str(n) => n.value.to_string(),
                  };
                  if let Some(src) = &export_named.src {
                    export_map.push((
                      name_str.clone(),
                      src.value.to_string(),
                      "*".to_string(),
                    ));
                  } else if self.wildcard {
                    export_map.push((
                      name_str.clone(),
                      "".into(),
                      "*".to_string(),
                    ));
                  } else {
                    is_barrel = false;
                    break;
                  }
                }
                ExportSpecifier::Named(s) => {
                  let orig_str = match &s.orig {
                    ModuleExportName::Ident(n) => n.sym.to_string(),
                    ModuleExportName::Str(n) => n.value.to_string(),
                  };
                  let name_str = match &s.exported {
                    Some(n) => match &n {
                      ModuleExportName::Ident(n) => n.sym.to_string(),
                      ModuleExportName::Str(n) => n.value.to_string(),
                    },
                    None => orig_str.clone(),
                  };

                  if let Some(src) = &export_named.src {
                    export_map.push((
                      name_str.clone(),
                      src.value.to_string(),
                      orig_str.clone(),
                    ));
                  } else if let Some((src, orig)) =
                    local_idents.get(&orig_str)
                  {
                    export_map.push((
                      name_str.clone(),
                      src.clone(),
                      orig.clone(),
                    ));
                  } else if self.wildcard {
                    export_map.push((
                      name_str.clone(),
                      "".into(),
                      orig_str.clone(),
                    ));
                  } else {
                    is_barrel = false;
                    break;
                  }
                }
                _ => {
                  if !self.wildcard {
                    is_barrel = false;
                    break;
                  }
                }
              }
            }
          }
          ModuleDecl::ExportAll(export_all) => {
            export_wildcards.push(export_all.src.value.to_string());
          }
          ModuleDecl::ExportDecl(export_decl) => {
            // Export declarations are not allowed in barrel files.
            if !self.wildcard {
              is_barrel = false;
              break;
            }

            match &export_decl.decl {
              Decl::Class(class) => {
                export_map.push((
                  class.ident.sym.to_string(),
                  "".into(),
                  "".into(),
                ));
              }
              Decl::Fn(func) => {
                export_map.push((
                  func.ident.sym.to_string(),
                  "".into(),
                  "".into(),
                ));
              }
              Decl::Var(var) => {
                let ids = collect_idents_in_var_decls(&var.decls);
                for id in ids {
                  export_map.push((id, "".into(), "".into()));
                }
              }
              _ => {}
            }
          }
          _ => {
              if !self.wildcard {
                // Other expressions are not allowed in barrel files.
                is_barrel = false;
                break;
              }
          }
      }
    }
    ModuleItem::Stmt(stmt) => match stmt {
      Stmt::Expr(expr) => match &*expr.expr {
        Expr::Lit(l) => {
          if let Lit::Str(s) = l {
            if allowed_directives && s.value.starts_with("use ") {
              directives.push(s.value.to_string());
            }
          } else {
            allowed_directives = false;
          }
        }
        _ => {
          allowed_directives = false;
          if !self.wildcard {
            is_barrel = false;
            break;
          }
        }
      },
      _ => {
        allowed_directives = false;
        if !self.wildcard {
          is_barrel = false;
          break;
        }
      }
    },
  }
}

Non-Wildcard Mode

In non-wildcard mode, it strictly requires that all exports must be in the form of re-exporting from other modules.

In non-wildcard mode (i.e., self.wildcard = false), the logic to determine whether a module conforms to barrel files specification is as follows:

1. When a namespace export (ExportSpecifier::Namespace) occurs:

   // ❌ No namespace export without source file, determine module does not conform to `barrel files` specification.
   import * as utils from './utils';
   export { utils };
   
   // ✅ Namespace export with source file is allowed, conform to `barrel files` specification.
   export * as utils from './utils';

2. When a named export (ExportSpecifier::Named) occurs:

   // ✅ Module first imports and then exports the same reference, conform to `barrel files` specification.
   import { foo } from './foo';
   export { foo };
   
   // ✅ Module re-exports other module's export, conform to `barrel files` specification.
   export { foo } from './foo';
   
   // ❌ Module context variable identifier export, does not conform to `barrel files` specification.
   const x = 42;
   export { x };

3. When an export declaration (ExportDecl) occurs:

   // ❌ Module context function declaration export, does not conform to `barrel files` specification.
   export function foo() {}
   
   // ❌ Module context class declaration export, does not conform to `barrel files` specification.
   export class MyClass {}
   
   // ❌ Module context variable declaration export, does not conform to `barrel files` specification.
   export const x = 42;

4. Other module declarations:

   // ❌ Module context default export declaration, does not conform to `barrel files` specification.
   export default function () {}

5. Non-string literal expressions:

   // ❌ Normal expression, does not conform to `barrel files` specification.
   console.log('hello');
   
   // ❌ Non-directive (e.g. `"use strict"`) literal, does not conform to `barrel files` specification.
   42;

In summary, in non-wildcard mode, a legal barrel file can only contain:

• Use export statement to re-export other modules.
• Use import statement to first import and then export.
• Use "use strict" directive string.

Wildcard Mode

In wildcard mode, it allows more flexible export patterns.

1. Namespace Export (ExportSpecifier::Namespace):

   // ✅ Namespace re-exports source module, conform to `barrel files` specification.
   export * as utils from './utils';
   
   // ✅ Conform to `barrel files` specification.
   import * as helpers from './helpers';
   export { helpers };

2. Named Export (ExportSpecifier::Named):

   // ✅ Named re-exports source module, conform to `barrel files` specification.
   export { foo } from './foo';
   
   // ✅ First imports and then exports, conform to `barrel files` specification.
   import { bar } from './bar';
   export { bar };
   
   // ✅ Named re-exports module declaration identifier, conform to `barrel files` specification.
   const x = 42;
   export { x };
   
   // ✅ Renamed exports module declaration identifier, conform to `barrel files` specification.
   export { x as myValue };

3. Export Declaration (ExportDecl):

   // ✅ Module context function declaration export, conform to `barrel files` specification.
   export function foo() {}
   
   // ✅ Module context class declaration export, conform to `barrel files` specification.
   export class MyClass {}
   
   // ✅ Module context variable declaration export, conform to `barrel files` specification.
   export const x = 42;
   export const y = 'hello';
   export var z = true;

4. Other module declarations:

   // ✅ Module context default export, conform to `barrel files` specification.
   export default function () {}
   
   // ✅ Wildcard export, conform to `barrel files` specification.
   export * from './utils';

5. Expressions and statements:

   // ✅ Normal expression, conform to `barrel files` specification.
   console.log('hello');
   
   // ✅ Literal, conform to `barrel files` specification.
   42;
   ('use strict');

In summary, the main features of wildcard mode are:

• Allow all types of exports (named, default, declaration, etc.).
• Allow module declarations and exports.
• Allow expressions and other statements.
• Do not require exporting from other modules.

This mode actually relaxes the definition of barrel file, allowing:

• Both as a re-export concentration module.
• Also include self module's export reference.
• Also include other code logic.

Rust Side Optimization Summary

If a module is determined to be non-barrel files, then next.js will not export anything.

if !is_barrel {
  new_items = vec![];
}

If a module is determined to be barrel files, then next.js will rewrite the module content, exporting the collected export_map and export_wildcards information.

if is_barrel {
  // Otherwise we export the meta information.
  new_items.push(ModuleItem::ModuleDecl(ModuleDecl::ExportDecl(ExportDecl {
    span: DUMMY_SP,
    decl: Decl::Var(Box::new(VarDecl {
      span: DUMMY_SP,
      kind: VarDeclKind::Const,
      decls: vec![VarDeclarator {
        span: DUMMY_SP,
        name: Pat::Ident(BindingIdent {
          id: private_ident!("__next_private_export_map__"),
          type_ann: None,
        }),
        init: Some(Box::new(Expr::Lit(Lit::Str(Str {
          span: DUMMY_SP,
          value: serde_json::to_string(&export_map).unwrap().into(),
          raw: None,
        })))),
        definite: false,
      }],
      ..Default::default()
    })),
  })));

  // Push "export *" statements for each wildcard export.
  for src in export_wildcards {
    new_items.push(ModuleItem::ModuleDecl(ModuleDecl::ExportAll(ExportAll {
      span: DUMMY_SP,
      src: Box::new(Str {
        span: DUMMY_SP,
        value: format!("__barrel_optimize__?names=__PLACEHOLDER__!=!{src}").into(),
        raw: None,
      }),
      with: None,
      type_only: false,
    })));
  }

  // Push directives.
  if !directives.is_empty() {
    new_items.push(ModuleItem::ModuleDecl(ModuleDecl::ExportDecl(ExportDecl {
      span: DUMMY_SP,
      decl: Decl::Var(Box::new(VarDecl {
        span: DUMMY_SP,
        kind: VarDeclKind::Const,
        decls: vec![VarDeclarator {
          span: DUMMY_SP,
          name: Pat::Ident(BindingIdent {
            id: private_ident!("__next_private_directive_list__"),
            type_ann: None,
          }),
          init: Some(Box::new(Expr::Lit(Lit::Str(Str {
            span: DUMMY_SP,
            value: serde_json::to_string(&directives).unwrap().into(),
            raw: None,
          })))),
          definite: false,
        }],
        ..Default::default()
      })),
    })));
  }
}

Take an example to explain

index.js

'use client';

import { foo as foo1 } from './foo';
export { bar } from './bar';
export * as utils from './utils';
export * from './utils2';
export { foo1 };

next.js will rewrite index.js to

rewritten index.js

export const __next_private_export_map__ = JSON.stringify([
  ['foo1', './foo', 'foo'],
  ['bar', './bar', 'bar'],
  ['utils', './utils', '*']
]);

export * from '__barrel_optimize__?names=__PLACEHOLDER__!=!./utils2';

export const __next_private_directive_list__ = JSON.stringify([
  'use client'
]);

This implementation collects all export methods through export_map and export_wildcards and uses is_barrel flag to determine whether the current module conforms to barrel files definition. In wildcard mode, it allows more flexible export patterns, while in non-wildcard mode, it strictly requires all exports to be in the form of re-exporting from other modules.

JS Optimization

In JS side, next.js implemented NextBarrelLoader plugin to optimize barrel files.

next-barrel-loader.ts

// code path: next.js/packages/next/src/build/webpack/loaders/next-barrel-loader.ts
const NextBarrelLoader = async function (
  this: webpack.LoaderContext<{
    names: string[];
    swcCacheDir: string;
  }>
) {
  // For barrel optimizations, we always prefer the "module" field over the
  // "main" field because ESM handling is more robust with better tree-shaking.
  const resolve = this.getResolve({
    mainFields: ['module', 'main']
  });

  const mapping = await getBarrelMapping(
    this.resourcePath,
    swcCacheDir,
    resolve,
    this.fs
  );
};

In the plugin, it calls getBarrelMapping to get the module's export information. If mapping is empty, it means that the module is not barrel files, and the subsequent optimizations end.

if (!mapping) {
  // This file isn't a barrel and we can't apply any optimizations. Let's re-export everything.
  // Since this loader accepts `names` and the request is keyed with `names`, we can't simply
  // return the original source here. That will create these imports with different names as
  // different modules instances.
  this.callback(null, `export * from ${JSON.stringify(this.resourcePath)}`);
  return;
}

Then the core logic of optimization is getBarrelMapping function, which is responsible for parsing the export information of barrel files and returning an object containing export information.

async function getBarrelMapping(
  resourcePath: string,
  swcCacheDir: string,
  resolve: (context: string, request: string) => Promise<string>,
  fs: {
    readFile: (
      path: string,
      callback: (err: any, data: string | Buffer | undefined) => void
    ) => void;
  }
) {
  // ...
  const res = await getMatches(resourcePath, false, false);
  barrelTransformMappingCache.set(resourcePath, res);

  return res;
}

In getBarrelMapping, it calls getMatches function to get the export information of barrel files. Of course, there's caching here to avoid repeated parsing.

async function getMatches(
  file: string,
  isWildcard: boolean,
  isClientEntry: boolean
) {
  if (visited.has(file)) {
    return null;
  }
  visited.add(file);

  const source = await new Promise<string>((res, rej) => {
    fs.readFile(file, (err, data) => {
      if (err || data === undefined) {
        rej(err);
      } else {
        res(data.toString());
      }
    });
  });

  const output = await transpileSource(file, source, isWildcard);
  // ...
}

Attention

Passed to rust side will include module path, source code, and whether to enable wildcard mode.

Source code is read through js side fs.readFile, this logic can be further optimized.

In getMatches, it calls transpileSource function.

// This is a SWC transform specifically for `optimizeBarrelExports`. We don't
// care about other things but the export map only.
async function transpileSource(
  filename: string,
  source: string,
  isWildcard: boolean
) {
  const isTypeScript =
    filename.endsWith('.ts') || filename.endsWith('.tsx');
  return new Promise<string>(res =>
    transform(source, {
      filename,
      inputSourceMap: undefined,
      sourceFileName: filename,
      optimizeBarrelExports: {
        wildcard: isWildcard
      },
      jsc: {
        parser: {
          syntax: isTypeScript ? 'typescript' : 'ecmascript',
          [isTypeScript ? 'tsx' : 'jsx']: true
        },
        experimental: {
          cacheRoot: swcCacheDir
        }
      }
    }).then(output => {
      res(output.code);
    })
  );
}

transpileSource function calls rust side's optimization logic for barrel files, returning a parsed product containing only the export statements of the module, details refer to Rust Side Implementation Details。

Attention

Note that the user specified barrel files file is used as the entry module for rust side parsing, not enabling wildcard mode (i.e., wildcard = false).

// The second parameter is false, indicating not to enable `wildcard` mode
const res = await getMatches(resourcePath, false, false);

Then follow through regular expression matching rust side returned product to get exportList, wildcardExports, isClientEntry information.

async function getMatches(
  file: string,
  isWildcard: boolean,
  isClientEntry: boolean
) {
  // ...

  const output = await transpileSource(file, source, isWildcard);
  const matches = output.match(
    /^([^]*)export (const|var) __next_private_export_map__ = ('[^']+'|"[^"]+")/
  );
  if (!matches) {
    return null;
  }

  const matchedDirectives = output.match(
    /^([^]*)export (const|var) __next_private_directive_list__ = '([^']+)'/
  );
  const directiveList = matchedDirectives
    ? JSON.parse(matchedDirectives[3])
    : [];
  // "use client" in barrel files has to be transferred to the target file.
  isClientEntry = directiveList.includes('use client');

  let exportList = JSON.parse(matches[3].slice(1, -1)) as [
    string,
    string,
    string
  ][];
  const wildcardExports = [
    ...output.matchAll(/export \* from "([^"]+)"/g)
  ].map(match => match[1]);

  // In the wildcard case, if the value is exported from another file, we
  // redirect to that file (decl[0]). Otherwise, export from the current
  // file itself.
  if (isWildcard) {
    for (const decl of exportList) {
      decl[1] = file;
      decl[2] = decl[0];
    }
  }

  // This recursively handles the wildcard exports (e.g. `export * from './a'`)
  if (wildcardExports.length) {
    await Promise.all(
      wildcardExports.map(async req => {
        const targetPath = await resolve(
          path.dirname(file),
          req.replace('__barrel_optimize__?names=__PLACEHOLDER__!=!', '')
        );

        const targetMatches = await getMatches(
          targetPath,
          true,
          isClientEntry
        );

        if (targetMatches) {
          // Merge the export list
          exportList = exportList.concat(targetMatches.exportList);
        }
      })
    );
  }

  return {
    exportList,
    wildcardExports,
    isClientEntry
  };
}

If rust side returned parsed product contains wildcard exports

export * from './utils';

It will continue to recursively call getMatches function

const targetMatches = await getMatches(
  targetPath,
  true,
  isClientEntry
)

Until all wildcard exports are parsed. At this point, you can notice that in parsing wildcard exports, the second parameter is true, indicating enabling wildcard mode.

Finally, after executing getBarrelMapping function, it returns the export and export path mapping relationship and instruction information of the user specified barrel files.

const NextBarrelLoader = async function (
  this: webpack.LoaderContext<{
    names: string[];
    swcCacheDir: string;
  }>
) {
  const mapping = await getBarrelMapping(
    this.resourcePath,
    swcCacheDir,
    resolve,
    this.fs
  );

  const exportList = mapping.exportList;
  const isClientEntry = mapping.isClientEntry;
  const exportMap = new Map<string, [string, string]>();
  for (const [name, filePath, orig] of exportList) {
    exportMap.set(name, [filePath, orig]);
  }
};

After obtaining the export and export path mapping relationship and instruction information of barrel files, the next step is to rewrite the import path of barrel files to specific module paths.

const NextBarrelLoader = async function (
  this: webpack.LoaderContext<{
    names: string[];
    swcCacheDir: string;
  }>
) {
  let output = '';
  const missedNames: string[] = [];
  for (const name of names) {
    // If the name matches
    if (exportMap.has(name)) {
      const decl = exportMap.get(name)!;

      if (decl[1] === '*') {
        output += `\nexport * as ${name} from ${JSON.stringify(decl[0])}`;
      } else if (decl[1] === 'default') {
        output += `\nexport { default as ${name} } from ${JSON.stringify(
          decl[0]
        )}`;
      } else if (decl[1] === name) {
        output += `\nexport { ${name} } from ${JSON.stringify(decl[0])}`;
      } else {
        output += `\nexport { ${decl[1]} as ${name} } from ${JSON.stringify(
          decl[0]
        )}`;
      }
    } else {
      missedNames.push(name);
    }
  }

  // These are from wildcard exports.
  if (missedNames.length > 0) {
    for (const req of mapping.wildcardExports) {
      output += `\nexport * from ${JSON.stringify(
        req.replace('__PLACEHOLDER__', missedNames.join(',') + '&wildcard')
      )}`;
    }
  }

  // When it has `"use client"` inherited from its barrel files, we need to
  // prefix it to this target file as well.
  if (isClientEntry) {
    output = `"use client";\n${output}`;
  }
};

This is the complete optimization logic of next.js for barrel files, including rust side optimization, js side optimization, and finally rewriting the import path of barrel files to specific module paths.

Summarize Optimization Of Next.js

1. next.js every time it communicates with rust side, it only parses the module once.

   • rust side will determine if the parsed module is the user specified entry barrel files module, then determine whether the module conforms to barrel files specification.
     • If not, stop current module barrel files optimization.
     • If yes, collect the module's export related information and return to js side.
   • rust side will determine if the parsed module is wildcard mode module, that is, this module is importer module through export * from ... re-exported module, then collect the module's export related information and return to js side.

2. next.js relies on rust side swc provided semantic analysis to accelerate parsing module import and export statements.

3. next.js will read the source code of the module to be parsed through js side fs.readFile and pass the source code as a string to rust side.

4. next.js for export * from ... wildcard re-export statement parsing

Feasibility Analysis

Above, we have understood the optimization logic of next.js, then let's do a runtime feasibility analysis.

Scene One: The dependent module is barrel files

According to the following example, barrel files dependent module foo itself is also barrel files.

main.js

import { bar } from '@/utils';
console.log(bar);

utils/index.js

export { bar } from './foo.js';

utils/foo.js

export { bar } from './bar.js';

utils/bar.js

export const bar = 'bar.js';

rollup.config.js

import barrelOptimize from '@rollup/plugin-barrel-optimize';

export default {
  input: 'src/main.js',
  output: {
    file: 'dist/bundle.js',
    format: 'esm'
  },
  plugins: [
    barrelOptimize([
      {
        entry: 'src/utils/index.js'
      }
    ])
  ]
};

next.js provides strategy that in configuration item (experimental.optimizePackageImports) specified as the entry module that needs optimization barrel files, that is utils/index.js module depends on utils/foo.js module, then utils/foo.js should also be included in experimental.optimizePackageImports to tell next.js that it needs to continue optimize utils/foo.js module.

When utils/foo.js module is not included in experimental.optimizePackageImports, then next.js will think utils/foo.js module is not barrel files, that is, in utils/foo.js module, there may exist side effect statements.

next.js parses re-exports only once for performance optimization

Regardless of how long the re-export chain of barrel files module is, next.js only parses the re-export statement once every time it communicates with rust side.

Example explanation:

// user code
import { bar } from '@/utils';
console.log(bar);

// utils/index.js
export { bar } from './foo.js';

// utils/foo.js
export { bar } from './bar.js';

// utils/bar.js
export const bar = 'bar.js';

In the above example, next.js will not recursively parse foo.js module, that is, it will not parse the second re-export module.

When parsing the second re-export module, since the user did not specify the second re-export module (foo.js) as barrel files in the configuration item, it means that the second re-export module (foo.js) may not only contain export statements but also side effect statements.

If you continue to recursively parse, it is likely to get undesired results.

Simple explanation:

foo.js module may exist in the following possibilities:

1. Non barrel files module
   1. Module exists side effects and re-exports other module's reference declaration identifier.
   2. Module does not exist side effects and re-exports other module's reference declaration identifier.
   3. Module exists side effects and exports module context's reference declaration identifier.
   4. Module does not exist side effects and exports module context's reference declaration identifier.
2. barrel files module
   1. Module re-exports other module's reference declaration identifier.

Only when the second re-export module (foo.js) is still barrel files module, next.js recursive parsing foo.js module has meaning. And confirm foo.js whether is barrel files module, repeat with configuration item (experimental.optimizePackageImports) function.

Possible question, if foo.js module is not barrel files module, then if this module does not exist side effects, then recursive parsing also has meaning, right?

Note that the current optimization stage is in building module dependency graph stage, which is before tree-shaking stage, so it cannot accurately determine whether foo.js module is side effects.

Possible doubt, can't we respect sideEffects configuration item?

If you have this doubt, then you haven't understood the meaning of sideEffects configuration item, detailed explanation refer to The Mechanism Of Module Side Effects, which explains in detail how sideEffects configuration item works in rollup.

Simply put, even if the module specifies sideEffects: false, although the first scan will not detect the side effects of this module (sideEffects: false), if the referenced identifier declared in the module (sideEffects: false) is used by other actual execution module, then it will reactivate the side effects detection of this module (sideEffects: false) the next scan, it will re-detect the side effects of this module (sideEffects: false) the next scan.

Scene Two: The dependent module has side effects

Above has mentioned that, next.js in optimizing barrel files when module dependency graph is not completed, so it does not have enough context information to semantic analysis barrel files dependent module whether exist side effects.

Below example exists with module execution order related side effects.

user code

import { foo } from '@/barrel-file';
console.log(foo);

barrel-file.js

export { bar } from './bar.js';
export { foo } from './foo.js';

foo.js

// moduleSideEffects: true
export const foo = 'foo.js';
console.log('side effect foo.js');

bar.js

// moduleSideEffects: true
export const bar = 'bar.js';
console.log('side effect bar.js');

If only rely on user actual used foo referenced identifier belonging module (foo.js), then bar.js module's side effects will not be detected. When foo.js and bar.js both identify exist side effects, this will cause the problem of inconsistent run result between development stage and production stage.

This is actually the problem vite is considering now, Consider treeshaking module for serving files in dev mode issue to the current stage has not been solved for the reason:

bluwy's reply

About this problem, I want to express my opinion.

• What makes it slow?

  Through some analysis, we found that the number of requests was not the exact indicator of speed. A js file importing 10000 original js modules, for vite, it parses speed is still fast.

  The real reason for slow is every module needs through plugin to execute transform, such as translating .ts module to .js module, .vue module to .js module. A large number of parsing work will cause vite internal to produce waterfall effect, that is, sequentially concurrently parsing all dependent modules of the module. Therefore, even if our team maintains the internal vite plugin fast enough, but third-party plugins may also have a huge impact on performance, and these third-party plugins are beyond our control.

• What can we do today

  One solution is to warm up some files so they are ready when needed: see #14291. This PR also explains why we choose this way. However, this feature currently does not cover warm-up for hot module replacement (HMR), but this will be implemented soon.

• About Tree-shaking

  vite-plugin-entry-shaking plugin effect is good, but I have reservations about side effects, because some barrel exported files may depend on these side effects. For barrel files module semantic analysis side effects cost is much higher than warm-up module cost.

  However, we cannot deny that most barrel files files do not have side effects, but there are still side effects scenarios. If it is used as a core feature, we still need to ensure its correctness. (In addition, this method may bring the risk of single instance repeated reference.)

• Conclusion

  Therefore, my opinion is that we can first see how #14291 effect on vite current stage without side effects optimization.

  If you want to find out which plugin executes slower through detection, you can run DEBUG="vite:plugin-*" vite dev way to implement, but please be cautious about these logs, because asynchronous code measurement is not always reliable, but it can still expose some more performance consuming work.

vite current stage is still in the unable to reasonably handle side effects and still exists barrel files performance problem. This is fatal for tools that depend on runtime like vite.

Official also warns Avoid Barrel Files, using a large number of barrel files will significantly increase vite performance overhead.

Summary Of Runtime Feasibility Analysis

In building dependency graph stage optimizing barrel files, due to lack of sufficient module context information, cannot accurately determine whether the module is side effects. Extra consideration of module side effects must exist boundary and performance problem. If you want to build dependency graph process to reasonably optimize barrel files, I believe that users need to ensure that every referenced identifier declared in the barrel files module are independent and most barrel files are actually independent modules, so for most users' application is conforming requirements.

Downstream tool vite or upstream tool rollup I think can clarify restrictions in official manual, implement for barrel files optimization, especially for tools like vite that depend on runtime, more important.

Contributors

XiSenao

Changelog

Last edited 21 days ago

View full history

9ba4ed7-feat: i18n

c28bec6-doc(material): shared translation notes material

64d2bfd-feat: reorganize document directory & change toolbox item to material item

c6e996c-chore: organize the directory structure of doc(rollup-optimizable-feature): barrel files article

bb2baac-doc(rollup-optimizable-feature): barrel files