Sunday, March 30, 2014

Reading Files During Dart Pub Transform


After last night, I have my post-Polymer.dart pub transformer on the road to building single-file JavaScript from Dart Polymer elements.

This transformer needs to run after the built-in Polymer.dart transformer and takes identical configuration in the pubspec.yaml file:
name: deployment_experiment
dependencies:
  polymer: any
dev_dependencies:
  scheduled_test: any
transformers:
- polymer:
    entry_points: web/index.html
- deployment_experiment:
    entry_points: web/index.html
Where the built-in Polymer.dart transformer leaves the <polymer-element> templates directly in the entry_points HTML, my deployment experiment transformer moves the templates into a new JavaScript file responsible for adding the templates to the DOM.

To complete my transformer, I need to combine the remaining <script> tags from the Polymer.dart transformer into a single script source:
<!DOCTYPE html>
<html lang="en">
<head>
  <script src="/scripts/x_pizza_templates.js"></script>
  <script src="/scripts/shadow_dom.min.js"></script>
  <script src="/scripts/custom-elements.min.js"></script>
  <script src="/scripts/interop.js"></script>
  <script src="/scripts/index.html_bootstrap.dart.js"></script>
</head>
<body>
  <div class="container">
    <h1>Ye Olde Dart Pizza Shoppe</h1>
    <x-pizza></x-pizza>
  </div>
</body>
</html>
The problem I am faced with is that none of those files is included as an asset coming from the Polymer.dart transform. So I cannot put my (admittedly small) pub transformer knowledge to use here. Regardless of how all of that JavaScript is going to get into my single “deploy.js” file, I will need to do the work near the buildSingleJavaScript() method of my transformer:
import 'package:barback/barback.dart';

class SingleJsPolymerTransformer extends Transformer {
  // ...
  buildSingleJavaScript(templateHtml, transform) {
    var templateId = new AssetId(
      'deployment_experiment',
      'web/scripts/deploy.js'
    );
    transform.addOutput(
      new Asset.fromString(templateId, _templateJs(templateHtml))
    );
  }
  // ...
}
This method created a new asset ID to be added to the transform process and it then adds the asset it from a string (which is the crazy JavaScript string version of the <polymer-element> tags from last night).

The absolute dumbest way that I can think to add all of the other JavaScript source files to this asset is read each synchronously from the file system, concatenating each to last night's _templateJS():
  buildSingleJavaScript(templateHtml, transform) {
    var templateId = new AssetId(
      'deployment_experiment',
      'web/scripts/deploy.js'
    );

    var js = _templateJs(templateHtml) +
      new File('packages/shadow_dom/shadow_dom.min.js').readAsStringSync() +
      new File('packages/custom_element/custom-elements.min.js').readAsStringSync() +
      new File('packages/browser/interop.js').readAsStringSync();

    transform.addOutput(new Asset.fromString(templateId, js));
  }
That works perfectly. At least it works perfectly until I try to read the dart2js generated Polymer code:
  buildSingleJavaScript(templateHtml, transform) {
    var templateId = new AssetId(
      'deployment_experiment',
      'web/scripts/deploy.js'
    );

    var js = _templateJs(templateHtml) +
      new File('packages/shadow_dom/shadow_dom.min.js').readAsStringSync() +
      new File('packages/custom_element/custom-elements.min.js').readAsStringSync() +
      new File('packages/browser/interop.js').readAsStringSync() +
      new File('build/web/index.html_bootstrap.dart.js').readAsStringSync();

    transform.addOutput(new Asset.fromString(templateId, js));
  }
When I try to pub build with that in my transformer, I am greeted with:
➜  dart git:(master) ✗ pub build
Building deployment_experiment...
Build error:
Transform SingleJsPolymer on deployment_experiment|web/index.html threw error: Cannot open file, path = 'build/web/index.html_bootstrap.dart.js' (OS E
rror: No such file or directory, errno = 2)
file_impl.dart 558                                                            _File.throwIfError
file_impl.dart 411                                                            _File.openSync
file_impl.dart 456                                                            _File.readAsBytesSync
file_impl.dart 479                                                            _File.readAsStringSync
http://127.0.0.1:46880/packages/deployment_experiment/transformer.dart 54:74  SingleJsPolymerTransformer.buildSingleJavaScript
...
The problem, of course, is that build/web/index.html_bootstrap.dart.js does not exist yet. This file is generated when pub build calls dart2js on *.dart code after the various transforms are invoked. But I need the code during a transform, so am I out of luck?

In fact, I may be out of luck. About the only thing that I can think to try is forking out a separate dart2js process during the transform. That seems ugly, but there may be no other option save for giving up the idea of doing this as a transformer and sticking with shell scripts. As wrong as the fork-dart2js-during-transform feels, I will likely give that a try tomorrow—at least to see just how ugly it is. Unless someone has a better idea?


Day #19

4 comments:

  1. dart2js has it's own transformer, which can be used in the pubspec.yaml file. Maybe you could include it, then include your transformer after it, and see if that will make your transformer happen after dart2js has done it's work. I know I reaching a bit here, but you never know.

    transformers:
    - polymer:
    entry_points:
    - web/index.html
    - $dart2js:
    minify: false
    - deployment_experiment:
    entry_points: web/index.html

    ReplyDelete
    Replies
    1. It's worth a try at least before I try forking a process. I believe that those are just options for dart2js, which is always run last. But that's more of an assumption so I'll check it out. Thanks!

      Delete
  2. Things have been updated, so now you can create a LazyTransformer, which can be made to run after dart2js like so

    transformers:
    - $dart2js
    - post_dart2js

    I got this from the [dart-misc] mailing list, after I asked a pointed question regarding this. It maybe to late for you, but I'll give it a go if you don't have time. I'll even try it if you do.

    ReplyDelete
  3. https://groups.google.com/a/dartlang.org/forum/#!topic/misc/mZnxRIIuR_Y

    ReplyDelete