At Agiledrop, we're always interested in new trends and technologies, especially those in the open source space. In 2020, we'll do our best to stay on top of things and inform our readers about exciting things we've encountered and worked on lately.
We recently got the chance to speak with Aaron Frost, founder of HeroDevs and ng-conf organizer, about a groundbreaking novelty for Angular - its first ever static site generator Scully. Read on to learn how the idea for Scully was born and implemented, what the top priorities and challenges were in development, and what we can expect from Scully going forward.
1. Tell us a little about yourself: your company, your role at the company and your role in the Angular community.
About a year and a half ago, I started a company called HeroDevs. We’re a small group of developers who work on architect-level contracting, and mostly we focus around Angular; we do a little bit of React as well, though.
So, that’s what I currently do. I also own another company called ng-conf, which is the world’s first Angular conference, and it’s a big adventure, it’s a completely different type of business to run.
As far as inside the Angular community, I’ve been a Google Developer Expert for about the last 6 years. And what that means is I had a lot of fun with AngularJS, and the Angular team invited me to do an interview to become a GDE and I passed that.
Being a GDE has made it so that I have an easier time teaching more people about Angular because Google will pay to fly you to different places around the world and have you give talks about Angular.
So, it’s largely changed my career for the better - that and being the ng-conf organizer are my 2 big roles in the Angular community. One of the things that I try to focus on is making the Angular community a fair place for everybody, not just for some of the people, but I try to make it as far for everybody as I can. That’s one of my big focuses.
If everyone felt as welcomed as me, that would make me happy - so that has got to be the goal!
2. How & when did the idea arise for a static site generator for Angular? What was the motivation behind it?
I think that the Angular community has wanted a static site generator for a very long time. I don’t think that I invented the concept at all; I remember 2 years ago being at ng-conf and hearing people say: “Man, I wish there was an Angular static site generator, I wish we had a blogging platform in Angular,” and I was hearing it.
And I was too busy at my job, I hadn’t started my own company yet, so I was too busy to say: “Hey, I’ll do that,” you know. And so I just let it go.
Then it was about 4 or 5 months ago that I revisited the thought, and I reached out to the Angular team, and I said: “Hey, Stephen - is this already being built? Or should I build it? Which one of the two? Should we help someone or should we build it from scratch?”
And Stephen said, “Well, we need it, and you should build it from scratch, but we have some ideas,” so I got on a phone call with him and we put our ideas together, and it seemed to work really well, just out of the gate, day one we made a lot of progress really quick.
So, Google pitched in on this, it wasn’t just me and my team. Two of the people on the Angular team that helped the most were Stephen, who helped us conceptualize it and kind of largely pull it together, and then the other member on the team is Minko; he built a tool called Guess.js, which makes part of the job that Scully has to do much easier.
A very high level of cohesion between the Angular team and the HeroDevs team was needed to get Scully to where it is today, which is really really exciting.
That’s pretty much where it came from; at first, our team, we kind of knew what static site generation was. We didn’t understand the benefits yet so we started building it without fully understanding why people would want it, but it quickly became apparent how awesome it was going to be - and I mean really quickly.
We started getting really really excited about what we could do with Scully. So, yeah, Scully is a really big deal for the Angular community. Over the next few years, certainly everyone reading your blog will have a chance to try Scully.
It’s going to make huge waves, and it’s going to change a lot of things for Angular apps. It’s a very exciting time.
3. What were the top priorities when developing Scully? What about some of the biggest challenges you encountered?
The top top priority was probably that it has to be easier than Angular Universal, because we already had Angular Universal and, from my experience, it’s too difficult. Like, I’m a smart developer, and Angular Universal costs me a lot of work, so what about all the newer developers? Are they going to be able to do it? I don’t think so.
And what about all the projects that already exist in Angular? Writing an Angular Universal app from scratch - that’s hard, but converting an already existing non-Universal app to be Universal is very difficult. So, we needed it to be easier than Universal no matter what we did.
For this reason, we used Puppeteer, because it solved the big problem with Universal, which is - your Angular has to render inside of Node, right, if you let it render inside of Puppeteer, then that’s just Chrome. And every Angular app should be able to render inside Chrome.
So we knew that by rendering to Puppeteer instead of Node, we were going to be able to get a much broader audience - including knowing that we would be able to expand to React and Vue and AngularJS, and reach a much wider level of audience.
Scully is going to be the static site generation platform in a year or two, it’ll be the one that people flock to, as it will render all JavaScript apps. And you’ll have one set of plugins that services the whole entire Angular community - that’s the really exciting thing about Scully.
We’re still hitting our heads on a couple challenges, though - one is the development experience, it needs to continue to get dialed in. The way Scully works right now is, when you run ng build you get a minified build version of your app, right?
Well, Scully depends on that to run, and that’s fine, except that every time you change your Angular code, you now need to redo a new ng build, which is a little bit problematic - well, it’s more than a little bit, it’s problematic.
We’re trying to get it to where you don’t have to do a build and then a Scully build every time, we want to get it to where you can just do the Scully build and not have to redo an ng build every single time.
So, we’re trying to tighten up the screws around the development experience - we have this thing called watch mode that we want to work, it’s just it hasn’t ever been conceptualized like we wanted to though, so there have definitely been some difficulties around developing this thing.
4. What are some of your favorite things about Scully?
One night I was just sitting and thinking when I realized that we had an opportunity to cache, not only to pre build your website, but to cache a lot of the calls that you make to your backend.
So I built this feature called TransferState, highly inspired from Angular Universal, which took care of that. During the time that I build your app with Scully, your Angular code needs to go get the data to show, right? So maybe your app shows tweets, or maybe it shows donuts, I don’t know - it shows something though, some kind of data from an API.
And, with just a couple extra lines of code, I could make your app fetch the data when you’re building your app with Scully and cache that. Then when it runs out in production it doesn’t actually need to go call the input anymore, cause it already cached the data during the build.
What that means is in production your app can stop making a lot of the calls that it currently makes out to the APIs, and so, if you write it the right way, you’re able to cache 100% of your non-dynamic data points.
Obviously, I can’t cache for example the shopping cart, cause the shopping cart’s dynamic, right, but I could cache the information on the blog, I could cache any sort of product information that’s on the page, any sort of links to other products.
There’s a lot of caching you can do, so that’s kind of my favorite feature, the amount of API calls you’re able to save by using Scully.
5. How does Scully compare to other static site generators - most notably to Gatsby for React?
It compares really well to Gatsby for React. Obviously Gatsby started a few years ago, so to compare them is brutal because we’re still in an alpha phase.
Once we have a few more months to catch up and to build some more feature parity, it’ll be pretty brutal, and here’s why: when you write a Gatsby app, you’re writing a very specific flavor of React, it’s not just a simple create-react-app with whatever router you’d like and so on, it’s a very Gatsby-specific app.
And you have to follow the rules otherwise it’s not going to work. That’s very much like Angular Universal: it’s a very specific flavor of Angular, it’s not all Angular. Same with Gatsby, it’s not all React - you can’t make an already existing React app be Gatsby, it’s very difficult to convert something that isn’t Gatsby into Gatsby.
Well, we wanted to mitigate that, so that with Scully, we don’t care, it’s just a browser app. Once we support React, we’ll support a Gatsby app, but we’ll also support a Create React App, or we’ll create an app that someone spun up on their own in React, we won’t really care.
So, I think we serve a much wider audience than what either Gatsby or Angular Universal currently serve.
The other really powerful thing is - one of the predictable things about Gatsby is that all the plugins are written on top of GraphQL, and that’s great - it means you have to learn GraphQL to write a plugin, however.
Scully plugins are just promise-based at this point, they don’t have any sort of limitation around “you have to learn GraphQL before you can integrate with Scully”. With Scully, you just write an Angular app, you don’t have to learn GraphQL to write this app.
It’s the same exact Angular app that you would’ve written without Scully, but now Scully knows to statically generate the site with it. It’s a very very powerful paradigm when compared to forcing all the users to learn GraphQL, so I think it compares extremely well to Gatsby - but I am biased. :)
Right now, it’s very Angular-specific; I’ve squeezed an Angular/AngularJS hybrid app in, and I think I could probably squeeze in a Vue or React app too, but it would definitely not be what you would consider first class support for React.
We need to get to that spot where React apps go “Oh yeah - all I had to do was install this thing and it just worked,” and we’re certainly not there today for React and Vue - but we will be there.
Another interesting thing that you haven’t specifically asked but I’m going to just volunteer is the name, Scully. So my company is called HeroDevs, so we like to name our projects with hero names, right?
When I was a kid, I really really liked the X-Files; the other thing I said I really care about was we wanted to name this thing a hero name, but we wanted to give it a strong female protagonist name, not just something nonsensical or some name that everyone knows.
We very intentionally wanted it to be a powerful female hero name. And, Dana Scully - the name just sounds right for this kind of project, X-Files and static files, generating, and so on - everything fit well; so, Dana Scully is the inspiration for the name.
As for branding and the logo - we actually have a logo sync this morning with our designer, and I’m pretty sure this will be the final logo. We’ve had quite a few back and forths in the last 2 months, and it’s wildly simple, yet I think it’s a logo people will remember - they’ll see it and be like: “That’s Scully.”
That’s important for me - that we didn’t just go all in on aliens, or all in on UFOs or whatever. I want Scully to be its own brand, and we can bring in the X-Files later. So, we have a strong brand, and we’re going to unveil it any day now. [Author's note: I talked with Aaron just a bit before the logo became public.]
6. What can we expect from Scully going forward - can you give us a sneak peek into any new features and when we might be getting them?
Jorge is working on the watch mode right now; once we have a more solid watch mode, we get a much quicker build time which will be really important. Then one of the other features that we’re trying to nail down is we’re trying to get it to where you can filter down to just do some of your routes.
So, when you do a build, let’s say you had a thousand distinct routes, but you didn’t want to rebuild all one thousand, maybe only 10% of them - maybe just the blog portion of your routes, you don’t want to rebuild everything else.
We’re trying to get in a filtering syntax, so you can say “only do these routes” - that’s going to be one of the next features. I think, for big companies, this will dramatically speed up their build time.
There’s one thing you do have to understand about a build with a static site: there are 2 things you have to consider with the build, whereas before you only had to consider one.
Before, you only had to consider if there was any new code, that’s it - if there’s new code, then we’ll do a build. With a Scully app, a static site generation app, you have to consider a) “is there new code?”, but you also have to consider b) “is there new content?”
Because you now need to do builds more often, both when there’s new code and also when there’s new content. The cool thing is, though, if there’s no new code, just new content, then you don’t have to rebuild everything; you have to rebuild a lot of stuff, but not everything.
So, you can build pieces of your app when it’s just content that changes, you don’t have to rebuild the whole thing. Imagine a company sells sunglasses or something, and 50% of their website is those sunglass pages, and the other 50% of their website are other marketing and blog pages.
Well, if none of the Angular has changed, just the sunglasses data has changed, then they can afford to only build half of the website, they don’t have to rebuild the whole entire website.
They don’t have to generate every single page, so if they have a hundred thousand pages, it might take them 20 minutes to generate that website, and that would be a significant build time.
But if you could cut down from 20 to 10, that would be huge, right, a huge savings for a business of that size, and so that’s one of the features that we’re trying to get on next.
But, ultimately, the thing that we’re really excited about - we have a really big client who’s given us permission to implement Scully for them, and they make lots of money, they’re a very very wealthy company.
And they’ve promised that we could have all the data about how much their conversion goes up when Scully is there, and so we’ve got a really cool blog post coming about how much money you can make off of using Scully in your Angular apps.
That’s something I’m probably the most excited about, we could end up on Hacker News; if these guys said “Hey, we spent $5000 to implement this and now we’re going to make $200,000 a year”, well then that’s like everyone’s brain just exploded, it’s like the development team just funded an architect, you know. It’s crazy to find that much money just kind of lying on the couch.
If anyone has any questions, we have something that we put a bit of pride into - if anyone needs any help, there are a couple different places you can go. We have tags on Stack Overflow so you can go out there and tag your questions with “Scully”.
We have a Gitter channel, usually someone’s on that pretty much 24 hours a day, so if you have any questions you should be able to get someone from the community pretty quickly on the Gitter channel, and then we also have our documentation on GitHub, which is in READMEs and so.
I would say that between the documentation, between our Gitter channel and Stack Overflow, there’s a high level of probability that someone’s going to be able to help you out quickly.
So, if you need anything you want to learn about it, head over to scully.io and all the contact and 24-hour information is just sitting right there in our documentation.
7. Apart from Scully, is there anything else in the Angular community that you’re excited about this year?
I’m excited to see what the community will do with Ivy. I think that the Angular team is excited to have Ivy out, and I think that having Ivy out enables them and the community to do some more innovating around Angular.
An example is Dominic Elm, he wrote this thing called Template Streams which allows you to with a very simple syntax create click handlers that are observable streams, which is very cool, because while I could’ve done that before, it took me some code to manually wire up. But Dominic now has that down to just a very simple syntax - and that’s only because of Ivy.
And then Sander Elias, for example he has it to where we can have observable lifecycle hooks, so instead of having ngOnInit as a function, I could have an OnInit observable, or I could have an OnChanges observable.
So, that’s my biggest thing: I’m not excited about Ivy for the sake of Ivy, I’m excited about Ivy for that Ivy unlocks and enables the community to do.
In itself, Ivy’s great, but it’s not meant for the developers, it’s meant to be hidden from us; it isn’t itself a tangible milestone that I can do anything with, but people who author libraries can do a lot of cool stuff now that Ivy’s here.
Anyway - my Twitter handle is @aaronfrost, my DMs are open - if anyone has any questions, feel free to reach out to me.
