Minutes of the 22nd meeting of the Scala Center, Q3 2021

Minutes are archived on the Scala Center website.

Summary

The following agenda was distributed to attendees: agenda.

Lukas Rytz is the new representative from Lightbend (replacing Adriaan Moors). He’s been a Scala team member at the company for seven years, and is now team lead; before that he got his Ph.D in Martin’s lab.

Claire McGinty is the new representative from Spotify (replacing Julien Tournay). She’s been a data engineer at Spotify for the last four years and her team maintains Scio, an open-source Scala library for Apache Beam.

The board elected a new chair, Chris Kipp. We customarily rotate this position annually. Seth Tisue served as acting chair for this meeting only.

There were no staffing changes at the Center this quarter.

Center activities for the past quarter focused on Google Summer of Code, the inclusive language guide, getting started with Coursier, MOOCs, the Scala documentation website, the Let’s Talk About Scala 3 video series, Scala in universities, the Scala Debug Adapter, Metals, bloop, Gradle, Scalafmt, Scalafix, sbt, Scaladex, Scastie, the Scala 3 compiler, the TASTy manipulation library, Scala.js, Scala Native, Scala Symposium, and communication and management.

Details on all this are in the directors’ quarterly report.

One new proposal was discussed:

  • SCP-027: Refactoring (Eugene Yokota (Twitter))

The proposal was sent back for revisions, for a possible vote at the next meeting.

Other business discussed included Scala 2, Bloop, the future of the Scala website, and community feedback.

Date, Time and Location

The meeting took place virtually on Wednesday, November 10, 2021 at 3:00pm (UTC).

Minutes were taken by Seth Tisue (secretary).

Attendees

Officers:

  • Seth Tisue (secretary, acting chairperson), Lightbend
  • Darja Jovanovic (executive director), EPFL
  • Sébastien Doeraene (technical director), EPFL
  • Martin Odersky (technical advisor), EPFL

Board members:

  • Diego Alonso, 47 Degrees
  • Maureen Elsberry, 47 Degrees
  • Graham Griffiths, Goldman Sachs
  • Chris Kipp, Lunatech
  • Kris Mok, Databricks
  • Rob Norris, community/Typelevel
  • Krzysztof Romanowski, VirtusLab
  • Lukas Rytz, Lightbend
  • Daniela Sfregola, Morgan Stanley
  • Claire McGinty, Spotify
  • Bill Venners, community/Artima
  • Eugene Yokota, Twitter

Apologies:

  • Nicolas Rémond, SwissBorg (affiliate member)

Guests:

  • Jamie Thompson, Scala Center
  • Raúl Raja Martinez, 47 Degrees

Activities report

Darja and Seb summarized Scala Center activities since the last meeting.

Their remarks were largely based on their quarterly report:

The following notes do not repeat the content of the report, but only supplement it.

Google Summer of Code: this was the first year with the Center overseeing it (instead of Martin’s lab LAMP, as in past years). Darja said that the Center intends to participate again next year, hopefully with even more students next time. (This year, there were four, “all very successful.”)

Darja thanked 47 Degrees and VirtusLab for their help with the “Let’s Talk about Scala 3” video series.

Chris Kipp asked when the work on making Coursier install the official sbt launcher (rather than an alternate with some different behaviors) would actually ship. Seb said a nightly build is already available, and the official release would be this quarter, possibly as soon as “several weeks” from now.

Seth asked about scala-cli, how does that interact with the Coursier work (SCP-026)? Seb: “scala-cli will be one of the applications Coursier installs”; users will still run cs setup, and one of the things they’ll get is scala-cli. (Perhaps even under the name scala, eventually.)

Elections

For chairperson, Chris Kipp put his name forward and was elected by acclaim. (We customarily rotate this position annually.)

Proposals

SCP-027: Refactoring

The proposal was submitted by Eugene Yokota (representing Twitter).

Eugene summarized the proposal. He emphasized that Twitter hoped that any solution should include Scala 2.

He also explained that Scalafix in itself isn’t sufficient as it stands. The ease of use it provides is not comparable to, for example, the refactoring support in IntelliJ and Metals.

On the proposal’s pull request, Seb already provided some feedback, beginning:

Our main concern with this proposal, as it stands, is that it defines a specific solution without clearly stating the problem […]

Eugene acknowledged this concern and offered to revise the proposal to be more centered on the “user story”, which “is what we’re most interested in.” The solution should be build tool neutral and IDE neutral, but should have some sort of UI.

Seb said the board could vote on the current version if it chooses, but suggested waiting and voting on a revised version.

Krzysztof expressed discomfort with voting on it at this meeting, based on feedback from his own team.

Seth pointed out that because this meeting was late, the next meeting will be held in two months, so the next opportunity to vote is sooner than usual.

Martin: “I personally find refactoring very interesting.” He contrasted “syntactic” and “formatting” rewrites with “deeper” ones. The latter kind are more difficult to perform correctly; that’s why IDEs give the user the chance to make corrections to suggested changes. If the desire is to perform deep changes that are known to be correct, that’s research. “You need to start with a semantically correct model, which TASTy would be but (for example) semanticdb is not. And you need a query language, which Scala 3 reflection could become,” and something like Datalog on top of it. It’s a “multi-year research project which I would personally find very exciting” but perhaps too ambitious for the Center.

Chris, Eugene, and Martin discussed possible designs that might or might not involve Scalameta, Scalafix, and/or BSP. BSP could be a way to provide build tool neutrality; Scalafix could be a reference implementation but not hardwired, for example if JetBrains wanted to swap in something else.

Voting: Given the feedback from the Center, we did not vote on this version of the proposal.

Other business

Scala 2

Eugene brought up Scala 2 and requested on Twitter’s behalf that news about Scala 2 be included in the Center’s technical report (even though much of the actual work on Scala 2 is led by Lightbend rather than by the Center). Seb said sure, we’ll coordinate with Lightbend on that. And after the meeting, Lukas agreed that his team would provide that as a section in future reports.

Kris seconded the continuing strong interest in Scala 2 on behalf of Databricks, and he also specifically asked about the issue of Java-serialization compatibility between minor Scala 2 versions. “What are the policies for merging changes” affecting compatibility? Databricks was recently affected by a change to serialization compatibility in Scala 2.13.7.

Claire also seconded the strong interest in Scala 2. Spotify still has a great deal of code running on Scala 2.12 that it will take time to migrate. (Lukas noted that going from 2.12 to 3 involves getting on 2.13 first.)

Krzysztof also agreed that there are many users to whom Scala 2 will remain important for some time; he talks to such users all the time.

Lukas said that changes to the Scala 2 language are strictly limited, for compatibility reasons. And “the Scala 2 standard library can only evolve in binary compatible ways,” but “serialization compatibility between minor releases was never promised.”

Seth described Lightbend’s process for taking feedback on upcoming Scala 2 releases. “A month or two before a release, we open a thread on contributors.scala-lang.org saying a release is upcoming, here are the changes merged so far, here are the remaining blockers,” and so on. “Our hope is that the board, and the whole community, will see those threads and use them, and test nightly builds, which are always available, and raise your hands and speak up before release time, if you want to. We’re open to making alterations, but that’s the current system.”

Bloop

Chris asked about the maintenance status of Bloop, since both Metals and now scala-cli rely upon it.

Seb: “Currently it receives the amount of maintenance that it needs in order to work with new versions of things, but there aren’t more plans for that, such as new features or other fundamental changes.”

Krzysztof elaborated further. He said the Bloop codebase has technical debt that makes it “really challenging to move forward”. Even just normal maintenance “takes quite a lot of time.” But “Bloop is too important now, so we cannot leave it in that state. Work will continue,” but the time frame for getting the project in a better state overall is unclear.

Rob: “Must Metals and scala-cli use Bloop? Is it a possibility to replace it?” He pointed out that Metals can talk directly to sbt now, via BSP. Krzysztof’s answer covered a lot of complex and interrelated ground, but one major point was performance (Bloop is very fast and it would take time to match that) and another was the desire, at present, not to hold up work on scala-cli.

Scala website

Darja preceded Jamie’s remarks with some history. The last major rework of the website was in 2013, though of course much maintenance has occurred since then. She also offered to have a separate meeting devoted entirely to this, so interested board members could attend and have more time to discuss improvements.

Jamie’s main points were as follows:

  • The scala-lang.org website does not represent the current state of Scala
  • No party is responsible for keeping the landing page of Scala current
  • Some new users may not identify what Scala solves for them
  • We propose that a named party become responsible for evolving the website
  • Some changes can be immediate, others are more of an investment.

The Center proposes a round of “quick fixes” in Q1 2022, followed by major changes for delivery in Q3 2022.

Rob: “I am overjoyed that you are looking at this and see a path forward. I talk to a lot of beginners who bump into these problems with the website, so I’m very excited about this.”

Kris mentioned that Spark website changes are in progress (e.g. to change from comparing with other tools, to proclaiming its own merits standalone). He offered to share some of Databricks’ experience with this.

Eugene recommended including “why Scala?” type of changes in the first, “quick fix” round, rather than holding it until later. He suggested Rust, Bazel, and sbt as sites that do a good job of making their value propositions clear.

Community feedback

Rob: “Watching public forums, I’m seeing many more questions being posed about Scala 3, and I’m seeing many more answers being given in Scala 3. It’s starting to tip, is my sense. That’s good to see.”

Rob also mentioned the issue of better coordinating the Scala 2 release process with releases of downstream tooling. There was a recent discussion at with participants including Lightbend, VirtusLab, Frank Thomas (Scala Steward), and other stakeholders. “I think this is a positive thing for the community.” Link:

  • https://contributors.scala-lang.org/t/coordinate-compiler-and-tooling-releases/5312

Darja added that the Center met with the IntelliJ team to discuss also coordinating Scala changes with them. “No decisions” yet, but they’re talking. Should there be a “tooling committee”, to better stay in sync?

Conclusion

The agenda accidentally omitted the usual “Financial report” section. We should be sure not to skip it next time.

Since this meeting occurred somewhat late, we should aim to meet again in mid-January. (And if the following meeting comes at the end of March, that will put us back on schedule.)