Minutes of the 9th meeting of the Scala Center, Q2 2018

Minutes are archived on the Scala Center website.

Summary

The following agenda was distributed to attendees: agenda.

Scala Center activities for the past quarter focused on Bloop, BSP, pipelined compilation, scalac-profiling, Zinc, Scalameta, SemanticDB, Scalafix, Dotty, Scala Native, sbt load-plugin, collections, scalajs-bundler, Accessible Scala, MOOCs, talks, open source sprees, conferences, and the Scala Improvement Process (SIP).

Full details on these activities are in Heather’s report.

Other topics discussed at the meeting included Scala 2.13 and Scala 3 migration, enforcing coding style guidelines, progress on tooling, web site maintenance, and metaprogramming plans for Scala 3.

No new proposals were made this quarter.

Date, Time and Location

The meeting took place at Morgan Stanley’s offices in New York City, on Tuesday, June 19, 2018 at 1:00pm local time. Most attendees were present in person but several participated virtually via conference call.

Minutes were taken by Seth Tisue (secretary).

There was no recording of the meeting this time, so I had to rely on my notes. I apologize for any resulting omissions or inaccuracies.

Attendees

Board members present:

James Belsey, Morgan Stanley
Eugene Burmako, Twitter
Thomas Gawlitza, SAP
Lars Hupel, community/Typelevel
Adriaan Moors, Lightbend
Juan Pedro Moreno, 47 Degrees
Jonathan Perry, Goldman Sachs
Frederick Reiss, IBM
Bill Venners, community/Artima

Also:

Sébastien Doeraene, EPFL

Officers:

Heather Miller (director), EPFL
Jon Pretty (chairperson), Propensive
Martin Odersky (technical advisor), EPFL
Seth Tisue (secretary), Lightbend

Proceedings

As chairperson, Jon Pretty conducted the meeting.

Activities

As Executive Director, Heather Miller summarized Scala Center activities since the last meeting.

Heather’s remarks were based on her detailed report on the Center’s recent activities.

The report includes links to a number of blog posts written by Scala Center engineers about their recent activities and collaborations.

The following notes are a supplement to Heather’s report.

The Bloop work will also be presented at at Scala Days.

Julien is working with Roland Kuhn (former Lightbend) and Konrad Malawski (Lightbend) on reviving the reactive MOOC (covering topics including typed actors and reactive streams).

Bill asked if the MOOCs still popular. Heather said yes, “way more popular than ever”, in fact. Making the courses available anytime, not just on particular start dates, really helped the numbers.

Seth asked if enough students been through the OpenEdX versions of the courses to be sure they are as solid as the Coursera versions. Heather said those numbers are only in the hundreds so far. Seth asked if the Center plans to push people more towards OpenEdX. Heather said no, it’s more of a backup plan, we aren’t trying to discourage Coursera use.

Eugene asked why the old collections didn’t go through a deprecation cycle first, with an intermediate period with both versions remaining available in at least one Scala version. Adriaan and Seth explained that doing it that way simply wasn’t practical. Jon pointed out that since the collections changes are the centerpiece of 2.13, other kinds of disruption were kept to a minimum in 2.13. It’s a “library release”, rather than a “compiler release” like 2.12.

Heather mentioned that in the “Accessible Scala” demo video linked from the notes uses “the Ubuntu voice” which “isn’t that good”, she said the voice on Mac is a lot better.

Scala Spree “is gaining momentum”, one hosted by Tapad was happening down the street literally as we were meeting. Many Scala conferences this year have hosted Sprees. “A zillion… 40?” people came to the Scala Days one.

The Scala Contributors Summit happened the day after Scala Days in Berlin. Heather thinks the attendes arrived at some better understandings on collaborative work.

Jon expressed optimism about the recent Zinc and Bloop work resulting in noticeable speed gains on real projects.

Darja Jovanovic, assuming no glitch occurs in the hiring process at EPFL, will be continuing to work on SIP and SPP and the Sprees and so forth.

Financial report

There is an engineering hire in the works, expected to be completed in the fall.

There is no other change since other recent meetings. The Center neither lost nor gained advisory board members this quarter (since Verizon left last quarter).

Jon asked if we should increase the fee for advisory board members, to increase available funds. There was some concern whether for some members, this might cause delays and difficult internal negotiations. Adriaan thinks we should pursue it.

Heather observed that the current model is $50K one-size-fits-all; should we introduce gradations by company size? Jon pointed out that there is also a $15K tier; Oracle is currently sponsoring at that level, but that tier doesn’t include any voting rights. Jon asked if companies at the lower level could perhaps receive partial voting rights somehow. There was some inconclusive discussion on this. Jon suggested that someone put a proposal on this together in advance of the next meeting. Heather pointed out that some other organizations vote on allowing new members, and we might consider that.

Elections

All of the current officers stood for re-election. No one else stood. There was no discussion and all were re-elected by acclamation.

Community feedback

Lars left early but said he didn’t have any community feedback to relay.

Bill said people are concerned about Scala 2.13 and Scala 3 migration. Is there a Scalafix ruleset for the 2.13 collections Scalafix? Yes, there already is, and it will be improved further before the final 2.13 release. We’ll cover Scala 3 under “other business”.

Other business

Since there were no new proposals this quarter, the board used the meeting time to discuss a number of issues of general concern to the Scala community.

Enforcing uniform style

Bill raised a community question: how do you keep people in a project using the same style? This concerns companies who are considering adopting Scala, as well as those who are already using it.

Some points that were raised in the ensuing discussion included:

Heather: some big companies have style guides such as the scala-lang.org style guide, the Twitter one, the DataBricks one, and so on. James said that Morgan Stanley uses a minor variation on the scala-lang one.
Several Scala linters and reformatters are available to help with this.
Adriaan said there is a limit to solving this technically, though, it’s a social problem.
For linting, the compiler itself and the general movement is to standardize on Scalafix (and Scalafmt for formatting). Heather noted that Scalafix is flexible enough to support customized rulesets. Ólafur is eager to continue to promote and improve Scalafix’s capabilities as a linter and not just a rewriter.
Frederick: it goes beyond syntactic/surface features of the language Uniformity is important so that new people can parachute into a codebase.
Jon: there is a commercial offering from Codacy, with integration with GitHub. (But no one was sure what its current capability set is.)

Migration

Eugene raised the topic of Scala 3 migration. The ensuing discussion also covered Scala 2.13 and 2.14.

There are a number of new language features in the pipeline (union types, intersection types, implicit functions, and so on). New features can be appealing, but migration always has costs. Eugene’s question: in a corporate environment, how can users “sell” the upgrade to management?

Martin: TASTY means the end of binary compatibility problems, not just after Scala 3 migration, but during Scala 2 to 3 migration. Like Java where there’s one bytecode and that’s all that matters, in Scala TASTY will be supported by both Scala 2.14 and Scala 3, so you will be able to use 2.14 artifacts on Scala 3. “This should take a lot of the potential pain” out of the migration.

Eugene: what if something has disappeared from the standard library? (including going back to 2.11?)

The question of the wholesale replacement of the collections API in 2.13 came up again. Martin said this was a special one-time, thing.

James, after praising Scala’s overall attention to making these version upgrades smooth, explained Morgan Stanley’s approach to migration. They cross-compile to both the old and new Scala versions only during a relatively brief migration period, then “pull the plug” on the old version, rather than continuing to cross-compile for an extended period. He also mentioned they have developed a macro internally that allows compile-time ifelse during the migration period.

Seth noted that the scala-collection-compat library (which already exists, and will be improved further before the final 2.13 release) supports cross-building collections-using code between 2.11, 2.12, and 2.13.

Seth also noted the importance of the plan for Scala 2.14 and Scala 3 to share the same standard library. That sharing will greatly limit how painful that migration can possibly end up being.

There was some discussion of Scalafix. Should it actually be made part of your build, when cross-building? James thought the mechanics of that could be prohibitive. Jon wasn’t sure if he agreed. But none of us have concrete experience with this yet.

Bill said he knew of at least one company that’s still on 2.10 just because upgrading everything all at once is too hard. He thinks that this compile-time if-else thing that James mentioned could help.

Returning to the subject of Scala 3, Martin said that Guillaume Martres has already done a proof-of-concept of TASTY-generation-for-Scala-2. Adriaan affirmed that TASTY support will be a centerpiece of Scala 2.14, and that the Lightbend team expects to continue Guillaume’s work and incorporate it into an early 2.14 milestone.

Adriaan also noted that in general, 2.14 theme is alignment with Dotty, so also standardizing on the Dotty backend is a priority, so he hopes that the 2.14 and 3 compilers can share the same backend. If this can be achieved, it will also limit the scope of possible migration pain.

James noted that Morgan Stanley’s migration issues tend to be not with Scala itself, but with libraries, that is, waiting for them to become available for the new Scala version. TASTY should solve this, since once libraries are published for 2.14, they will be usable on Scala 3 as well.

Eugene returned to the question of how to “sell” the Scala 3 upgrade. Scala’s problem isn’t lack of features, he said, it’s lack of tools.

Martin noted that TASTY should also be a basis for improving the tooling situation; there is some material on this in his Scala Days Berlin slides. He also said the new compiler is “geared towards IDEs”. So Scala 3 promises, for example, a “much much better VSCode experience”, in fact “we have that already [in Dotty], we could never have done that with the Scala 2 presentation compiler.”

James raised the issue of compiler speed. If the Scala 3 compiler were slower than Scala 2 compiler, that would be a problem. With the improvements that Jason Zaugg and others have made and are still making to the speed of the Scala 2 compiler, Scala 2 “might pull ahead temporarily”, Martin said, but he expects they will be able to make comparable improvements to the Scala 3 compiler once the work on Scala 3 is farther along and the team is free to focus more on performance again.

Eugene asked how soon Scala 3 would be ready for initial testing on Twitter’s codebase. The requirements are that basics works and that the feature set has been frozen, so they aren’t testing a still-moving target, features-wise. Martin responded that by Scala Days 2019 a feature freeze should be in effect, with a final release targeted for early 2020.

Returning to Scala 2.13, Eugene asked for more time to try Scala 2.13.0-M4 before the 2.13 feature freeze. What if there are unfixable problems that can’t be fixed because of the feature freeze? Seth and Adriaan were able to reassure Eugene on the nature of the 2.13 freeze. It just means whole new data structures and new operations aren’t being added anymore. Bugs are still being fixed, minor adjustments are still being made, and the migration experience and the cross-building experience are still being improved.

Eugene asked what the recommendation is for cross-compiling to 2.13? Adriaan and Seth answered that there will be a migration guide on this, there is material already on collections migration, but there will be more. Better documentation is a major goal for 2.13.0-RC1 and 2.13.0, including more coverage on how to do cross-compilation.

Tooling

Someone asked about Metals. In a Scala Center context, it falls under Scalameta work. Heather said that Ólafur hopes to work in this in future months.

Eugene asked how Metals compares with the Dotty LSP implementation.

Martin said the plan is to auto-detect if something is a Scala 2 or 3 project, and then use Metals or Dotty LSP behind the scenes. There will be a single plugin in the marketplace that does both.

Jon asked about the LSP and STP working groups, how are they doing? Note that the working groups are making sure people are on the same page about compatibility and not duplicating effort, they aren’t tasked with actual project work.

Web sites

Seth asked about web site maintenance, scala-lang.org and associated sites. Could more of that be happening at the Scala Center, under the already submitted Scala Center proposal? Not sure what’s practical here.

Adriaan suggested we work harder to encourage and allow more community people to become committers and take more responsibility in the website repos. Heather and Seth seconded this.

Metaprogramming

The issue of “culture wars” and “clashes of styles” in the Scala community came up again. Martin expressed a hope that Scala 3 will “overcome the tension” by doing something that’s better than either side of some current conflicts. One of his goals for Scala 3’s features is to bring different styles together to reduce division in the community. An example of this is the improvements to typelevel programming that Miles Sabin is participating in.

Martin showed current work on implementing HList using the new transparent keyword. You don’t know how many type parameters you have, but it doesn’t matter, you don’t write any, transparent makes it happen. Example is an HList of length 5, xs(6) fails at compile time. It’s inlining-based, but inlining of (processed, not textual) source code. Design is new, implementation is fragile, but both are promising. There are three different implementation schemes being debated.

Jon: could you have transparent methods on StringContext? Martin: yes.

Eugene and Martin discussed the similarities and differences between whitebox macros and transparent. Martin is reluctant to allow running user-level code in typer, as in whitebox macros, so he wants to offer a “tamed” form where only certain transformations/simplifications are allowed. (But, see also https://github.com/lampepfl/dotty/pull/5382.)

Heather: this stuff keeps changing, how do I keep up? Martin: for now, you have to look at the pull requests. Heather: is it too soon to suggest this become a proto-SIP? A single document that lays out the design. Martin: it’s coming. We’ll get there. This is where we foresee the biggest changes (in Scala 3), so yes, we’ll putting this out for feedback. “We need to push this out quickly so people can try this out with their macro libraries.”

Conclusion

The next meeting will be virtual and will be held in approximately three months.