The Hashing Dilemma

Related Materials

[v3.0] New hashing algorithm that "fixes (nearly) everything" - GitHub

[v3.0] Use ASCII characters for hash placeholders - GitHub

The hashing dilemma, Rollup 3, and our future with Vite | Lukas Taegert Atkinson | ViteConf 2022 - YouTube

In rollup version v3.0, the hash algorithm was refactored, introducing a new hash algorithm that resolves long-standing hash instability issues, properly handles renderChunk plugin transformations, and supports circular dependencies.

Problem Statement

The execution flow of the old version's hashing algorithm was as follows:

1. Render all modules, excluding dynamic imports and import.meta chunk references

   • First generate the main code of modules
   • But at this point, dynamic imports (like import('./foo.js')) and chunk references in import.meta.url don't know their final filenames yet, so they are skipped

2. Calculate the content hash for all modules in each chunk based on the above

   • Generate hash values using module contents

3. Extend the hash: consider all known dependencies + potential dynamic content added to the chunk wrapper

   • Chunk wrapper refers to the formatting code that wraps module content (such as import/export statements for ES modules, require/exports for CommonJS)
   • These potentially changing contents also need to be included in the hash calculation

4. Update dynamic imports and import.meta chunk references

   • Only now are the actual chunk filenames filled in

5. Render chunk wrappers (containing all static imports and exports)

6. Process the result through the renderChunk plugin hook

Existing Issues:

1. renderChunk Plugin Hook Breaks content hash

   Any transformations in renderChunk are completely ignored by rollup and do not affect the chunk's hash value. This leads to situations where different contents can have the same hash.

2. Chunk wrapper Not Included in hash Calculation

   Having rollup maintain every change in the chunk wrapper to extend hash changes requires considering too many edge cases.

The fundamental problem with the old algorithm is that the hash doesn't reflect the final output content. Since renderChunk is the core processing stage for many plugins (like terser minification), changes in this part are completely ignored, making the hash unreliable as a cache identifier.

---

Old Algorithm Source-Level Analysis (Before PR #4543)

Source baseline: commit 9216f5235^ (the commit before PR #4543 was merged)

Complete Hash Composition

The old algorithm's hash consisted of a two-layer structure:

Layer 1: getRenderedHash() — Content hash of a single chunk

// src/Chunk.ts (v2.x) Lines 524-551
getRenderedHash(): string {
  if (this.renderedHash) return this.renderedHash;
  const hash = createHash();

  // ① Return value from augmentChunkHash plugin hook
  const hashAugmentation = this.pluginDriver.hookReduceValueSync(
    'augmentChunkHash',
    '',
    [this.getChunkInfo()],
    (augmentation, pluginHash) => {
      if (pluginHash) {
        augmentation += pluginHash;
      }
      return augmentation;
    }
  );
  hash.update(hashAugmentation);

  // ② Module code after preRender() (MagicStringBundle concatenation)
  hash.update(this.renderedSource!.toString());

  // ③ Export name mapping: moduleId:variableName:exportName
  hash.update(
    this.getExportNames()
      .map(exportName => {
        const variable = this.exportsByName.get(exportName)!;
        return `${relativeId((variable.module as Module).id)}:${
          variable.name
        }:${exportName}`;
      })
      .join(',')
  );

  return (this.renderedHash = hash.digest('hex'));
}

Layer 2: computeContentHashWithDependencies() — Complete hash with dependencies

// src/Chunk.ts (v2.x) Lines 886-908
private computeContentHashWithDependencies(
  addons: Addons,
  options: NormalizedOutputOptions,
  bundle: OutputBundleWithPlaceholders
): string {
  const hash = createHash();

  // ④ intro + outro + banner + footer
  hash.update([addons.intro, addons.outro, addons.banner, addons.footer].join(':'));

  // ⑤ Output format
  hash.update(options.format);

  // Traverse self and all transitive dependencies (static + dynamic)
  const dependenciesForHashing = new Set<Chunk | ExternalModule>([this]);
  for (const current of dependenciesForHashing) {
    if (current instanceof ExternalModule) {
      // ⑥ External module path
      hash.update(`:${current.renderPath}`);
    } else {
      // ⑦ Content hash of dependency chunks (recursively calls getRenderedHash)
      hash.update(current.getRenderedHash());
      // ⑧ Filename template of dependency chunks (excluding hash part)
      hash.update(current.generateId(addons, options, bundle, false));
    }
    if (current instanceof ExternalModule) continue;
    for (const dependency of [...current.dependencies, ...current.dynamicDependencies]) {
      dependenciesForHashing.add(dependency);
    }
  }

  return hash.digest('hex').substr(0, 8);
}

Hash Composition Summary Table

No.	Content	Source Method	Description
①	augmentChunkHash plugin return	getRenderedHash()	User plugins can inject
②	Module code after rendering	getRenderedHash()	Contains original dynamic import paths
③	Export name mapping	getRenderedHash()	moduleId:varName:exportName
④	intro:outro:banner:footer	computeContentHashWithDependencies()	addons content
⑤	Output format string	computeContentHashWithDependencies()	options.format
⑥	External module paths	computeContentHashWithDependencies()	ExternalModule.renderPath
⑦	①②③ of all dependency chunks	computeContentHashWithDependencies()	Recursive getRenderedHash()
⑧	Filename template of dependency chunks	computeContentHashWithDependencies()	Excluding hash part

Content NOT in the Hash

Content	Reason
import/export statements (wrapper code)	finalise() executes in render(), after hash calculation
Final chunk paths for dynamic imports	finaliseDynamicImports() executes in render()
Final value of import.meta.url	finaliseImportMetas() executes in render()
Modifications from renderChunk hook	Executes at the end of render()

Source Evidence of Execution Timing

Evidence 1: Bundle.generate() Call Order

// src/Bundle.ts Lines 60-78
async generate(isWrite: boolean): Promise<OutputBundle> {
  // ...
  // 1. Pre-render all chunks
  this.prerenderChunks(chunks, inputBase, snippets);

  // 2. Assign chunk IDs (triggers hash calculation)
  await this.addFinalizedChunksToBundle(chunks, inputBase, addons, outputBundle, snippets);
  // ...
}

// src/Bundle.ts Lines 85-103
private async addFinalizedChunksToBundle(...) {
  // assignChunkIds calls generateId → computeContentHashWithDependencies
  this.assignChunkIds(chunks, inputBase, addons, bundle);

  // render happens after hash calculation
  await Promise.all(
    chunks.map(async chunk => {
      Object.assign(outputChunk, await chunk.render(...));
    })
  );
}

Evidence 2: ImportExpression.render() Doesn't Render Final Paths

// src/ast/nodes/ImportExpression.ts Lines 48-76
render(code: MagicString, options: RenderOptions): void {
  // ...
  if (this.mechanism) {
    // Only renders the mechanism parts of import( and )
    code.overwrite(..., this.mechanism.left, ...);
    code.overwrite(..., this.mechanism.right, ...);
  }
  // this.source.render() renders the original path, like './foo.js'
  // not the final chunk path
  this.source.render(code, options);
}

// renderFinalResolution is a separate post-processing method
// called in Chunk.render() → finaliseDynamicImports()
renderFinalResolution(
  code: MagicString,
  resolution: string,  // This is where the final chunk path is
  ...
): void {
  code.overwrite(this.source.start, this.source.end, resolution);
}

Evidence 3: Post-Processing Order in Chunk.render()

// src/Chunk.ts Lines 710-711
async render(...): Promise<{ code: string; map: SourceMap }> {
  // ... other processing ...

  // These two methods execute in render(), by which time the hash is already calculated
  this.finaliseDynamicImports(options, snippets);  // Fill in final dynamic import paths
  this.finaliseImportMetas(format, snippets);      // Fill in import.meta.url

  // ... finalise() generates wrapper ...

  // renderChunk hook executes last
  let code = await renderChunk({
    code: prevCode,
    ...
  });

  return { code, map };
}

Distinguishing Chunk Wrapper from Export Name Mapping

It's important to distinguish between two concepts:

Aspect	Export Name Mapping (in hash ③)	import/export Statements (not in hash)
Nature	Metadata string	Actual JavaScript code
Example	"src/utils.js:foo:foo"	export { foo };
Generation Timing	Available after preRender()	render() → finalise()
Contents	Module path, variable name, export name	Complete syntax, paths, format-specific code

Practical Impact:

Since the export name mapping already captures the semantic information of exports (what's exported, from which module), the main impact of import/export statements not being in the hash is:

• Order changes in import statements won't change the hash
• The same export expressed with different syntax (theoretically) won't change the hash

But export name changes (like foo → bar) will change the hash because ③ contains this information.

---

Solution

Sorted Rendering (Attempting to Solve Hash Issues)

One method to solve hash issues is to first render chunks that have no dependencies, then iteratively render chunks that only depend on already rendered chunks, until all chunks are rendered. While this approach works in some cases, it has several obvious drawbacks:

1. Doesn't Support Circular Dependencies Between Chunks

   This is a very important feature, as in this context, circular dependencies could also be two chunks that dynamically import each other. Additionally, rollup heavily relies on a mechanism when handling dynamic imports:

   rollup moves all shared dependencies between the dependent chunk and the dependency chunk to the dependent chunk, resulting in a static import of the dependent chunk in the dependency chunk.

   Mechanism Explanation

   Suppose there are three modules, module main (entry module), module b, and module c, where:

   • Module main dynamically imports module b and statically imports module c.
   • Module b dynamically imports module main and statically imports module c.

   // main.js
   import { c } from './c.js';
   console.log('a.js');
   import('./b.js').then(res => {
     console.log(res, c);
   });

   ---

   // b.js
   import { c } from './c.js';
   console.log('c.js');
   import('./main.js').then(res => {
     console.log(res, c);
   });

   ---

   // c.js
   console.log('c.js');
   export const c = '123';

   In this scenario, rollup will move the shared static dependencies (module c) between module main and module b to module main. This means there will be a static import of module main in module b.

   // main.js
   console.log('c.js');
   const c = '123';
   
   console.log('a.js');
   import('./Ckpwfego.js').then(res => {
     console.log(res, c);
   });
   
   var main = /*#__PURE__*/ Object.freeze({
     __proto__: null
   });
   
   export { c, main as m };

   ---

   // Ckpwfego.js
   import { c } from './main.js';
   
   console.log('c.js');
   import('./main.js')
     .then(function (n) {
       return n.m;
     })
     .then(res => {
       console.log(res, c);
     });

   The above mechanism of rollup ensures that when dynamically importing, all dependencies shared with the dynamic import have already been loaded. For calculating necessarily-loaded atoms for dynamic entry D and removing dependency markers, see Chunk Assignment for detailed explanation.

2. The sorted rendering chunk algorithm means that before rendering a chunk, we need to understand all its dependencies, and also consider that the renderChunk hook might introduce new dependencies.

Hash Placeholders

Therefore, a new solution needs to be introduced. The core idea is to set initial placeholders for filename references, so that the calculated hash value is independent of filenames and only focuses on the chunk's own content.

Execution flow is as follows:

1. Assign an initial filename to each chunk. If the filename does not contain a hash (no [hash] placeholder in options.chunkFileNames), this will be the final filename; but if the filename contains a hash, use an equal-length placeholder instead.

   rollup/src/Chunk.ts

   class Chunk {
   
     private preliminaryFileName: PreliminaryFileName | null = null;
   
     getPreliminaryFileName(): PreliminaryFileName {
       if (this.preliminaryFileName) {
         return this.preliminaryFileName;
       }
       let fileName: string;
       let hashPlaceholder: string | null = null;
       const {
         chunkFileNames,
         entryFileNames,
         file,
         format,
         preserveModules
       } = this.outputOptions;
       if (file) {
         fileName = basename(file);
       } else if (this.fileName === null) {
         const [pattern, patternName] =
           preserveModules || this.facadeModule?.isUserDefinedEntryPoint
             ? [entryFileNames, 'output.entryFileNames']
             : [chunkFileNames, 'output.chunkFileNames'];
         fileName = renderNamePattern(
           typeof pattern === 'function'
             ? pattern(this.getPreRenderedChunkInfo())
             : pattern,
           patternName,
           {
             format: () => format,
             hash: size =>
               hashPlaceholder ||
               (hashPlaceholder = this.getPlaceholder(
                 patternName,
                 size || DEFAULT_HASH_SIZE
               )),
             name: () => this.getChunkName()
           }
         );
         if (!hashPlaceholder) {
           fileName = makeUnique(fileName, this.bundle);
         }
       } else {
         fileName = this.fileName;
       }
       if (!hashPlaceholder) {
         this.bundle[fileName] = FILE_PLACEHOLDER;
       }
       // Caching is essential to not conflict with the file name reservation above
       return (this.preliminaryFileName = { fileName, hashPlaceholder });
     }
   
     getFileName(): string {
       return this.fileName || this.getPreliminaryFileName().fileName;
     }
   
     getImportPath(importer: string): string {
       return escapeId(
         getImportPath(
           importer,
           this.getFileName(),
           this.outputOptions.format === 'amd' && !this.outputOptions.amd.forceJsExtensionForImports,
           true
        )
      );
     }
   }

   rollup/src/utils/hashPlaceholders.ts

   // Four random characters from the private use area to minimize risk of
   // conflicts
   const hashPlaceholderLeft = '!~{';
   const hashPlaceholderRight = '}~';
   const hashPlaceholderOverhead =
     hashPlaceholderLeft.length + hashPlaceholderRight.length;
   // This is the size of a 128-bits xxhash with base64url encoding
   const MAX_HASH_SIZE = 21;
   export const DEFAULT_HASH_SIZE = 8;
   export const getHashPlaceholderGenerator =
     (): HashPlaceholderGenerator => {
       let nextIndex = 0;
       return (optionName, hashSize) => {
         if (hashSize > MAX_HASH_SIZE) {
           return error(
             logFailedValidation(
               `Hashes cannot be longer than ${MAX_HASH_SIZE} characters, received ${hashSize}. Check the "${optionName}" option.`
             )
           );
         }
         const placeholder = `${hashPlaceholderLeft}${toBase64(
           ++nextIndex
         ).padStart(
           hashSize - hashPlaceholderOverhead,
           '0'
         )}${hashPlaceholderRight}`;
         if (placeholder.length > hashSize) {
           return error(
             logFailedValidation(
               `To generate hashes for this number of chunks (currently ${nextIndex}), you need a minimum hash size of ${placeholder.length}, received ${hashSize}. Check the "${optionName}" option.`
             )
           );
         }
         return placeholder;
       };
     };

2. Render all modules in the chunk. Since we already have the initial filename from step 1, we can directly render all dynamic imports and import.meta chunk references. The old algorithm calculated the chunk content hash separately from the dynamic import chunk hash and import.meta chunk hash, then calculated them together again. The new algorithm calculates the hash only once, and subsequent modifications to the hash value are related to the chunk's content, not the filename.

3. Render the chunk wrapper, also using the initial filename to handle chunk imports.

   Purpose of chunk wrapper

   Essentially, the chunk wrapper operation is key to forming interop between chunks.

   Because a single chunk is rendered from one or multiple modules, during rendering, rollup replaces import/export statements between modules with specific references from the modules (e.g., converting import to direct references to exported variables).

   However, between chunks, rollup (or users through the splitChunks plugin configuration) performs further optimization on the chunk graph, potentially building new chunk dependencies (dynamic imports or static imports). Therefore, rollup needs to use the chunk wrapper operation to form interop between chunks, ensuring the completeness of the dependency chain.

   rollup/src/Chunk.ts

   class Chunk {
     async render(): Promise<ChunkRenderResult> {
       const { intro, outro, banner, footer } = await createAddons(
         outputOptions,
         pluginDriver,
         this.getRenderedChunkInfo()
       );
       finalisers[format](
         renderedSource,
         {
           accessedGlobals,
           dependencies: renderedDependencies,
           exports: renderedExports,
           hasDefaultExport,
           hasExports,
           id: preliminaryFileName.fileName,
           indent,
           intro,
           isEntryFacade:
             preserveModules ||
             (facadeModule !== null && facadeModule.info.isEntry),
           isModuleFacade: facadeModule !== null,
           log: onLog,
           namedExportsMode: exportMode !== 'default',
           outro,
           snippets,
           usesTopLevelAwait
         },
         outputOptions
       );
       if (banner) magicString.prepend(banner);
       if (format === 'es' || format === 'cjs') {
         const shebang =
           facadeModule !== null &&
           facadeModule.info.isEntry &&
           facadeModule.shebang;
         if (shebang) {
           magicString.prepend(`#!${shebang}\n`);
         }
       }
       if (footer) magicString.append(footer);
     }
   }

   rollup/src/finalisers/es.ts

   export default function es(
     magicString: MagicStringBundle,
     {
       accessedGlobals,
       indent: t,
       intro,
       outro,
       dependencies,
       exports,
       snippets
     }: FinaliserOptions,
     {
       externalLiveBindings,
       freeze,
       generatedCode: { symbols },
       importAttributesKey
     }: NormalizedOutputOptions
   ): void {
     const { n } = snippets;
   
     const importBlock = getImportBlock(
       dependencies,
       importAttributesKey,
       snippets
     );
     if (importBlock.length > 0) intro += importBlock.join(n) + n + n;
     intro += getHelpersBlock(
       null,
       accessedGlobals,
       t,
       snippets,
       externalLiveBindings,
       freeze,
       symbols
     );
     if (intro) magicString.prepend(intro);
   
     const exportBlock = getExportBlock(exports, snippets);
     if (exportBlock.length > 0)
       magicString.append(n + n + exportBlock.join(n).trim());
     if (outro) magicString.append(outro);
   
     magicString.trim();
   }

4. Process the chunk through the renderChunk hook.

   The new algorithm also allows access to the complete chunk graph in the renderChunk plugin hook, although at this point the names are initial placeholders. However, since rollup makes no assumptions about the output of renderChunk, you can now freely inject chunk names in this hook.

   rollup/src/utils/renderChunk.ts

   const chunkGraph = getChunkGraph(chunks);
   
   async function transformChunk(
     magicString: MagicStringBundle,
     fileName: string,
     usedModules: Module[],
     chunkGraph: Record<string, RenderedChunk>,
     options: NormalizedOutputOptions,
     outputPluginDriver: PluginDriver,
     log: LogHandler
   ) {
     const code = await outputPluginDriver.hookReduceArg0(
       'renderChunk',
       [
         magicString.toString(),
         chunkGraph[fileName],
         options,
         { chunks: chunkGraph }
       ],
       (code, result, plugin) => {
         if (result == null) return code;
   
         if (typeof result === 'string')
           result = {
             code: result,
             map: undefined
           };
   
         // strict null check allows 'null' maps to not be pushed to the chain, while 'undefined' gets the missing map warning
         if (result.map !== null) {
           const map = decodedSourcemap(result.map);
           sourcemapChain.push(
             map || { missing: true, plugin: plugin.name }
           );
         }
   
         return result.code;
       }
     );
   }
   
   function getChunkGraph(chunks: Chunk[]) {
      return Object.fromEntries(
        chunks.map(chunk => {
          const renderedChunkInfo = chunk.getRenderedChunkInfo();
          return [renderedChunkInfo.fileName, renderedChunkInfo];
        })
      );
    }

5. Calculate the pure content hash of the chunk by replacing all placeholders in the chunk with default placeholders and generating the hash.

   To ensure that the hash value is only related to the chunk's content itself and remains consistent across different builds, we need to replace the placeholders in the chunk with a fixed, identical value before calculating the hash. This way, the hash value won't be affected by the specific content of the placeholders, thus ensuring consistency and reproducibility.

   rollup/src/utils/hashPlaceholders.ts

   const REPLACER_REGEX = new RegExp(
     `${hashPlaceholderLeft}[0-9a-zA-Z_$]{1,${
       MAX_HASH_SIZE - hashPlaceholderOverhead
     }}${hashPlaceholderRight}`,
     'g'
   );
   export const replacePlaceholdersWithDefaultAndGetContainedPlaceholders =
     (
       code: string,
       placeholders: Set<string>
     ): { containedPlaceholders: Set<string>; transformedCode: string } => {
       const containedPlaceholders = new Set<string>();
       const transformedCode = code.replace(REPLACER_REGEX, placeholder => {
         if (placeholders.has(placeholder)) {
           containedPlaceholders.add(placeholder);
           return `${hashPlaceholderLeft}${'0'.repeat(
             placeholder.length - hashPlaceholderOverhead
           )}${hashPlaceholderRight}`;
         }
         return placeholder;
       });
       return { containedPlaceholders, transformedCode };
     };

6. Enhance the chunk's content-hash through the augmentChunkHash hook.

   rollup/src/utils/renderChunk.ts

    const { containedPlaceholders, transformedCode } =
      replacePlaceholdersWithDefaultAndGetContainedPlaceholders(code, placeholders);
    let contentToHash = transformedCode;
    const hashAugmentation = pluginDriver.hookReduceValueSync(
      'augmentChunkHash',
      '',
      [chunk.getRenderedChunkInfo()],
      (augmentation, pluginHash) => {
        if (pluginHash) {
          augmentation += pluginHash;
        }
        return augmentation;
      }
    );
    if (hashAugmentation) {
      contentToHash += hashAugmentation;
    }

7. After all chunks have completed their content-hash calculations, calculate the final hash by searching for which placeholders are contained in each chunk and updating the chunk's hash. Recursively retrieve the content-hash of all dependent chunks in the chunk and merge them to enhance the final chunk's content-hash.

   rollup/src/utils/renderChunk.ts

     function generateFinalHashes(
       renderedChunksByPlaceholder: Map<string, RenderedChunkWithPlaceholders>,
       hashDependenciesByPlaceholder: Map<string, HashResult>,
       initialHashesByPlaceholder: Map<string, string>,
       placeholders: Set<string>,
       bundle: OutputBundleWithPlaceholders,
       getHash: GetHash
     ) {
       const hashesByPlaceholder = new Map<string, string>(initialHashesByPlaceholder);
       for (const placeholder of placeholders) {
         const { fileName } = renderedChunksByPlaceholder.get(placeholder)!;
         let contentToHash = '';
         const hashDependencyPlaceholders = new Set<string>([placeholder]);
         for (const dependencyPlaceholder of hashDependencyPlaceholders) {
           const { containedPlaceholders, contentHash } =
             hashDependenciesByPlaceholder.get(dependencyPlaceholder)!;
           contentToHash += contentHash;
           for (const containedPlaceholder of containedPlaceholders) {
             // When looping over a map, setting an entry only causes a new iteration if the key is new
             hashDependencyPlaceholders.add(containedPlaceholder);
           }
         }
         let finalFileName: string | undefined;
         let finalHash: string | undefined;
         do {
           // In case of a hash collision, create a hash of the hash
           if (finalHash) {
             contentToHash = finalHash;
           }
           finalHash = getHash(contentToHash).slice(0, placeholder.length);
           finalFileName = replaceSinglePlaceholder(fileName, placeholder, finalHash);
         } while (bundle[lowercaseBundleKeys].has(finalFileName.toLowerCase()));
         bundle[finalFileName] = FILE_PLACEHOLDER;
         hashesByPlaceholder.set(placeholder, finalHash);
       }
       return hashesByPlaceholder;
     }

8. Replace placeholders with the final hash. Since equal-length placeholders were used in step 1, there's no source map position offset, so no need to update the source map.

   rollup/src/utils/renderChunk.ts

   import { replacePlaceholders } from './hashPlaceholders';
   
   function addChunksToBundle(
      renderedChunksByPlaceholder: Map<string, RenderedChunkWithPlaceholders>,
      hashesByPlaceholder: Map<string, string>,
      bundle: OutputBundleWithPlaceholders,
      nonHashedChunksWithPlaceholders: RenderedChunkWithPlaceholders[],
      pluginDriver: PluginDriver,
      options: NormalizedOutputOptions
   ) {
      for (const {
        chunk,
        code,
        fileName,
        sourcemapFileName,
        map
      } of renderedChunksByPlaceholder.values()) {
        let updatedCode = replacePlaceholders(code, hashesByPlaceholder);
        const finalFileName = replacePlaceholders(fileName, hashesByPlaceholder);
      }
   }

   rollup/src/utils/hashPlaceholders.ts

   const REPLACER_REGEX = new RegExp(
     `${hashPlaceholderLeft}[0-9a-zA-Z_$]{1,${
       MAX_HASH_SIZE - hashPlaceholderOverhead
     }}${hashPlaceholderRight}`,
     'g'
   );
   export const replacePlaceholders = (
     code: string,
     hashesByPlaceholder: Map<string, string>
   ): string =>
     code.replace(
       REPLACER_REGEX,
       placeholder => hashesByPlaceholder.get(placeholder) || placeholder
     );

To avoid accidental replacement of non-placeholders, placeholders utilize the feature of javascript supporting unicode characters. Random characters from the reserved plane are used, such as \uf7f9\ue4d3 (placeholder start) and \ue3cc\uf1fe (placeholder end).

Placeholder Transformation

[v3.0] Use ASCII characters for hash placeholders made improvements to placeholders to address the following issues:

1. Prevent Escaping Issues

   Using unicode characters can be automatically escaped in certain toolchains, causing placeholders to be corrupted.

2. Better Debugging Experience

   Compared to incomprehensible unicode characters, the new format uses visible ascii characters, making placeholders immediately recognizable, allowing developers to quickly identify their association with a particular chunk.

3. Reduce Risk of False Matches

   The new pattern _!~{\d+}~ is not valid javascript syntax and will only appear in strings and comments. Even if incorrectly replaced, it will only cause limited damage, as it will only be replaced when there is an exact match of the specific number sequence.

New Algorithm Source-Level Analysis

Source baseline: Current latest version of the rollup repository

Core Data Structures

The new algorithm introduces several key data structures to coordinate hash calculation:

// src/utils/renderChunks.ts Lines 27-30
interface HashResult {
  containedPlaceholders: Set<string>;  // Placeholders of other chunks referenced in this chunk's code
  contentHash: string;                  // This chunk's content hash (placeholders replaced with defaults)
}

// src/Chunk.ts Lines 83-91
interface PreliminaryFileName {
  fileName: string;      // Filename with placeholder, e.g., "chunk-!~{1}~.js"
  hashPlaceholder: string | null;  // Placeholder string, e.g., "!~{1}~"
}

Data Flow:

1. PreliminaryFileName is generated in step 1, containing the filename with placeholder
2. HashResult is calculated in steps 5-6, storing each chunk's content hash and its references to other chunks
3. generateFinalHashes() uses this data in step 7 to calculate the final hash

Circular Dependency Handling: Transitive Closure Algorithm

The generateFinalHashes() in step 7 is the core innovation of the new algorithm, solving circular dependencies through transitive closure:

// src/utils/renderChunks.ts Lines 293-329
function generateFinalHashes(...) {
  for (const placeholder of placeholders) {
    let contentToHash = '';
    // Initialize dependency set with only itself
    const hashDependencyPlaceholders = new Set<string>([placeholder]);

    // Key: Dynamically adding new elements while iterating over a Set
    // JavaScript Set feature: Elements added during iteration will be accessed in subsequent iterations
    for (const dependencyPlaceholder of hashDependencyPlaceholders) {
      const { containedPlaceholders, contentHash } =
        hashDependenciesByPlaceholder.get(dependencyPlaceholder)!;

      // Accumulate dependency content hashes
      contentToHash += contentHash;

      // Add other chunks referenced by this dependency to the set
      for (const containedPlaceholder of containedPlaceholders) {
        hashDependencyPlaceholders.add(containedPlaceholder);
      }
    }

    // Calculate final hash on accumulated content
    finalHash = getHash(contentToHash).slice(0, placeholder.length);
  }
}

Algorithm Analysis:

Assuming circular dependency A ↔ B (A references B, B also references A):

Step	hashDependencyPlaceholders	contentToHash
Init	{A}	""
Process A	{A, B} (found A references B)	contentHash_A
Process B	{A, B} (B references A, but A already exists)	contentHash_A + contentHash_B
End	Set no longer grows, loop terminates	Final value

Key Points:

• Set automatically deduplicates, avoiding infinite loops
• JavaScript's Set allows dynamic element addition during iteration
• Final hash = hash(concatenation of contentHash from all chunks in the dependency chain)

Hash Composition Comparison: Old vs New Algorithm

Component	Old Algorithm	New Algorithm
Module code after rendering (preRender phase)	✅	✅
Export name mapping	✅	✅ (included in rendered code)
augmentChunkHash plugin return value	✅	✅
intro/outro/banner/footer	✅	✅ (included in rendered code)
options.format	✅	✅ (reflected through wrapper)
External module paths	✅	✅ (included in rendered code)
Final paths for dynamic imports	❌	✅ (in placeholder form)
Final value of import.meta.url	❌	✅ (in placeholder form)
import/export statements (wrapper)	❌	✅
Modifications from renderChunk hook	❌	✅
Hash of all dependency chunks	Partial	✅ (transitive closure)

Complete Execution Flow (Source-Level)

Bundle.generate() [src/Bundle.ts:53-105]
  │
  ├─ getHashPlaceholder = getHashPlaceholderGenerator()
  │   └─ Create placeholder generator, format "!~{sequence}~"
  │
  ├─ generateChunks() → Generate all Chunk objects
  │   └─ Each Chunk calls getPreliminaryFileName() [src/Chunk.ts:580-616]
  │       └─ Returns { fileName: "chunk-!~{1}~.js", hashPlaceholder: "!~{1}~" }
  │
  └─ renderChunks() [src/utils/renderChunks.ts:40-89]
      │
      ├─ 1. reserveEntryChunksInBundle() → Reserve entry chunk filenames
      │
      ├─ 2. Promise.all(chunks.map(chunk => chunk.render()))
      │   └─ Chunk.render() [src/Chunk.ts:703-788]
      │       ├─ renderModules() → Render all module code
      │       ├─ finalisers[format]() → Generate wrapper (import/export statements)
      │       └─ Returns ChunkRenderResult (containing MagicString)
      │
      ├─ 3. transformChunksAndGenerateContentHashes()
      │   │   [src/utils/renderChunks.ts:202-291]
      │   │
      │   ├─ 3.1 Execute in parallel for each chunk:
      │   │   └─ transformChunk() → Call renderChunk plugin hook
      │   │
      │   ├─ 3.2 replacePlaceholdersWithDefaultAndGetContainedPlaceholders()
      │   │   └─ Replace "!~{1}~" with "!~{00000000}~"
      │   │   └─ Record containedPlaceholders (which other chunks are referenced)
      │   │
      │   ├─ 3.3 pluginDriver.hookReduceValueSync('augmentChunkHash', ...)
      │   │   └─ Plugins can inject additional content for hash calculation
      │   │
      │   └─ 3.4 getHash(contentToHash) → Calculate initial contentHash
      │       └─ Store in hashDependenciesByPlaceholder Map
      │
      ├─ 4. generateFinalHashes() [src/utils/renderChunks.ts:293-329]
      │   └─ Transitive closure algorithm: Merge all dependency contentHashes to calculate final hash
      │
      └─ 5. addChunksToBundle() [src/utils/renderChunks.ts:332-405]
          └─ replacePlaceholders() → Replace placeholders with final hash

New Algorithm Impact

Plugin Hook Execution Flow Diagram

The plugin hook execution flow diagram has changed. Here is the post-transformation flow diagram:

parallel

sequential

first

async

sync

Compared to the pre-transformation flow diagram:

The following changes have occurred:

1. Changes in Execution Timing

   • The execution timing of banner, footer, intro, and outro plugin hooks has been delayed. Previously, they were executed after the renderStart plugin hook. Now they are executed before the renderChunk plugin hook.
   • The execution timing of the augmentChunkHash plugin hook has been delayed. Previously, it was executed after the renderDynamicImport plugin hook decision. Now it is executed after the renderChunk plugin hook.

2. Changes in Execution Mode

• The banner, footer, intro, and outro plugin hooks have been changed from parallel execution to sequential execution.

Available Chunk Information in Hooks

Some hooks can now receive additional information. Before detailing these changes, let's define several key types:

PrerenderedChunk

PrerenderedChunk contains basic chunk information before any rendering occurs and before the chunk name is generated. After this update, this simplified chunk information is only passed to the entryFileNames and chunkFileNames options. From the new flow diagram above, we can see that at this stage it's impossible to obtain information about already rendered modules. As an alternative, it now includes a moduleIds list, allowing developers to at least roughly understand what's contained in the chunk.

interface PreRenderedChunk {
  exports: string[];
  facadeModuleId: string | null;
  isDynamicEntry: boolean;
  isEntry: boolean;
  isImplicitEntry: boolean;
  moduleIds: string[];
  name: string;
  type: 'chunk';
}

RenderedChunk

RenderedChunk contains complete rendering information for the chunk. The imports and filenames in rendered modules will contain placeholders rather than file hashes. RenderedChunk is available in the renderChunk hook, augmentChunkHash hook, and banner, footer, intro, outro hooks and options.

Additionally, the signature of renderChunk has been extended with a fourth parameter meta: { chunks: { [fileName: string]: RenderedChunk } }, providing access to the entire chunk graph.

Additional Points to Note

When adding or removing imports or exports in renderChunk, rollup will not do additional work to help maintain the RenderedChunk object. Therefore, user plugins should now be careful to maintain the RenderChunk object themselves, updating the latest RenderedChunk object information. This will provide correct information for subsequent plugins and the final bundle. Because later, rollup will replace imports, importedBindings, and dynamicImports placeholders based on the information in the RenderedChunk object to generate the final hash value (except for implicitlyLoadedBefore and fileName).

interface RenderedChunk {
  dynamicImports: string[];
  exports: string[];
  facadeModuleId: string | null;
  fileName: string;
  implicitlyLoadedBefore: string[];
  importedBindings: {
    [imported: string]: string[];
  };
  imports: string[];
  isDynamicEntry: boolean;
  isEntry: boolean;
  isImplicitEntry: boolean;
  moduleIds: string[];
  modules: {
    [id: string]: RenderedModule;
  };
  name: string;
  referencedFiles: string[];
  type: 'chunk';
}

New Features

• intro, outro, banner, footer as functions are now called for each chunk. Although they cannot access rendered modules in the chunk, they will receive a list of all moduleIds contained in the chunk.

• Hash length can be changed in the filename pattern, for example, [name]-[hash:12].js will create a hash with a length of 12 characters.

Breaking Changes

• entryFileNames and chunkFileNames cannot access the modules object that contains rendered module content. Instead, they can access the list of contained moduleIds.
• The order of plugin hooks has changed, please compare the above diagram with the diagram in the Rollup documentation.
• The fileName and referenced imports in the renderChunk hook will get filenames with placeholders instead of hashes. However, these filenames can still be safely used in the hook's return value, as any hash placeholder will eventually be replaced with the actual hash.

Test Cases

Online Demo Repository

main.js

import('./b.js').then(res => {
  console.log(res);
});

b.js

import('./c.js').then(res => {
  console.log(res);
});
export const qux = 'QUX';

c.js

export const c = 'c';

rollup.config.js

import { defineConfig } from 'rollup';

export default defineConfig({
  input: 'main.js',
  output: {
    dir: 'dist',
    format: 'es',
    chunkFileNames: '[hash].js'
  }
});

The bundled output is as follows:

main.js

import('./CM53L61n.js').then(res => {
  console.log(res);
});

CM53L61n.js

import('./CPjDz2XZ.js').then(res => {
  console.log(res);
});
const qux = 'QUX';

export { qux };

CPjDz2XZ.js

const c = 'c';

export { c };

If we only change the filename of b.js to bNext.js, keeping everything else the same:

Online Demo Repository

main.js

import('./bNext.js').then(res => {
  console.log(res);
});

bNext.js

import('./c.js').then(res => {
  console.log(res);
});
export const qux = 'QUX';

c.js

export const c = 'c';

rollup.config.js

import { defineConfig } from 'rollup';

export default defineConfig({
  input: 'main.js',
  output: {
    dir: 'dist',
    format: 'es',
    chunkFileNames: '[hash].js'
  }
});

The bundled output is as follows:

main.js

import('./CM53L61n.js').then(res => {
  console.log(res);
});

CM53L61n.js

import('./CPjDz2XZ.js').then(res => {
  console.log(res);
});
const qux = 'QUX';

export { qux };

CPjDz2XZ.js

const c = 'c';

export { c };

Through the above examples, we can see that in the new algorithm, file name changes do not cause the hash value of the chunk to change.

Contributors

XiSenao

Changelog

Last edited about 1 month ago

View full history

f521861-feat(rollup): update hashing dilemma with interactive visualization

3bb29bb-feat: i18n

3cf010c-doc(rollup-events): supplement the implementation details of hashing dilemma

0dd58f4-doc(rollup-events): adjust content structure and description

1b7993d-chore: delete useless markdownlint comment

7c11721-docs(rollup-events): supplement hash dilemma content

c0676ae-doc(rollup-events): hash dilemma