Minutes of the 10th meeting of the Scala Center, Q3 2018

Minutes are archived on the Scala Center website.

Summary

The following agenda was distributed to attendees: agenda.

Sébastien Doeraene is the new director of the Scala Center.

Scala Center activities for the past quarter focused on MOOCs, collections, scalajs-bundler, Bloop, pipelined compilation, Zinc, compiler performance, Dotty tooling and IDE, Scalafix, Scalameta, ScalaPB, conferences (including Scala Days), talks, open source sprees, and the Scala Improvement Process (SIP).

Full details on these activities are in Sébastien’s report.

Other topics discussed at the meeting included forum moderation and possible changes to how Center sponsorship works.

One new proposal was made:

• SCP-018: Converging the Intermediate Representation of Scala 2.14 and Scala 3.0 (Lightbend)
  • voted on and accepted

Date, Time and Location

The meeting took place virtually, via Google Meet, on Friday, September 21, 2018 at 4:00pm (GMT).

Minutes were taken by Seth Tisue (secretary).

Attendees

Board members present:

• James Belsey, Morgan Stanley
• Eugene Burmako, Twitter
• Chris Davenport, community/Typelevel (filling in for Lars Hupel)
• Adriaan Moors, Lightbend
• Jonathan Perry, Goldman Sachs
• Bill Venners, community/Artima

Apologies:

• Thomas Gawlitza, SAP
• Raúl Raja Martínez, 47 Degrees
• Frederick Reiss, IBM

Officers:

• Sébastien Doeraene (director), EPFL
• Jon Pretty (chairperson), Propensive
• Martin Odersky (technical advisor), EPFL
• Seth Tisue (secretary), Lightbend

Proceedings

As chairperson, Jon Pretty conducted the meeting.

Activities

As the Center’s new Executive Director, Sébastien Doeraene summarized Scala Center activities since the last meeting.

Most of Sébastien’s remarks were based on his detailed report on the Center’s recent activities.

The following notes are a supplement to Sébastien’s report:

Julien said the collaboration with Roland and Konrad on the reactive systems MOOC was “very effective” and he has been “very happy” with the results so far.

Jorge provided some preliminary performance numbers for build pipelining: Akka compiles 17% faster, Spark compiles 30% faster.

Ólafur says that the improved documentation in the Scalafix 0.8.0 release was praised on Twitter by a user who wrote some rules using the new release.

In addition to the Scala open source sprees covered in the report, a spree was also held in Gdańsk. This spree was notable because it happened with substantially less direct participation from the Scala Center than any previous spree (only Jon was physically present).

Scala Days 2019 will be held at EPFL in Lausanne in June. Because it’s at EPFL this time, Darja Jovanovic is doing much of the organization work, in collaboration with Lightbend, others at EPFL, and other parties.

Eugene asked about Martin Duhem’s work on Dotty IDE. He asked whether this work might also benefit Scala 2 users, and if so, when. Sébastien said Ólafur has a plan for this that he intends to work on next quarter, involving Metals and further work on Scalameta LSP.

Financial report

Sébastien said the budget is “pretty much stable”. As before, the seven advisory board members provide funding for three engineers, and the MOOCs fund a fourth.

The MOOC funds have also now paid back the Scala Center’s seed funds that were used to bootstrap creation of the center.

MOOC funds since January 2018 have yet to arrive, but most of that delay is normal. There has been some additional delay because of the changeover of directors.

Proposals

SCP-018: Converging the Intermediate Representation of Scala 2.14 and Scala 3.0

Proposed by Adriaan Moors on behalf of Lightbend.

See the proposal text for details.

Also, there was some discussion on the pull request for the proposal between Adriaan, Eugene, and Guillaume Martres.

In presenting the proposal to the board, Adriaan noted that a prototype implementation by Guillaume already exists, so the proposal is to put more resources behind moving that forward.

Eugene re-raised the same questions from his comments on the pull request. He suggested that the prototype should be improved for one quarter, as a trial, before the board commits any engineering resources for a longer period.

Jon asked how much engineering effort is available at the Center to work on this. Sébastien said that he’s not sure if the current engineering staff has enough compiler expertise to be fully effective on this work. He hopes to hire someone new for this.

Jon asked if the current engineers could be trained. Sébastien said yes, of course it’s possible, just not ideal.

Adriaan points out that this would be a collaborative effort with Lightbend and LAMP, so much of the compiler expertise would come from there. He believes the work could be divided appropriately among the involved parties to make this work, regardless of whom from the Scala Center is put on the project. (It helps that Adriaan is now based at the Lightbend office in Lausanne, which is across the street from the Scala Center.)

Adriaan hopes that the two compilers’ back ends could be aligned closely enough to produce the same bytecode for nearly all code. That isn’t the focus of this proposal, but the TASTY work will be easier the more the two back ends can be aligned.

James asked if the shared back end could ultimately be kept in a single shared repo. If the Scala 2 teams makes a back end change, how would they know if their change breaks Dotty? Adriaan said he could “definitely see” a shared repo happening, though it’s “on the more ambitious side of the spectrum” of possible outcomes. Perhaps a CI workflow involving multiple repos could be an equivalent solution.

Jon asked what would happen if the proposal were rejected, what interoperability picture between Scala 2 and 3 would result. Martin answered that without Scala Center help the story “looks bleak”, because LAMP cannot pick up the slack, and Lightbend cannot do all the work on this without delaying Scala 3 substantially, there is too much else to do for 2.14. Jon asked for clarification: “If we didn’t do this, that would mean that Dotty could depend on Scala 2 projects using Scala signatures, but not the other way around?” Adriaan said yes. Martin added that as a result “migration would have to be bottom up, you couldn’t migrate anything from Scala 2 to Scala 3 unless all your dependencies have migrated, and you’d need dual builds for the whole duration, because as long as you still had Scala 2 clients, you’d need a Scala 2 build, so it would complicate [ecosystem migration] greatly”. Sébastien agreed that doing this work would substantially decrease the risk of a difficult community transition to Scala 3.

Eugene asked if it would be feasible for the Scala 2 compiler to read TASTY but not generate it, would that remove the risk? Martin responded that reading them is actually harder than writing them. “The unpickler is larger than the pickler”, and “Scala 2 would have to deal with all the Dotty features”, at least well enough (Adriaan added) to link against them. Martin: “Once you’ve understood the necessary transformations, it’s relatively easy to do both reading and writing, and that would be useful for testing, for example that the round trip is the identity.” Adriaan added that both compilers still need to generate bytecode, so if both back ends aren’t TASTY based, that creates risks of incompatibilities.

Eugene asked to what extent the proposed Scala 2 TASTY writer would need to be exactly aligned with the TASTY the Scala 3 compiler outputs; harmonizing these outputs could be “the biggest complexity”? He added that he’s been facing similar issues in the work on rsc at Twitter, and he’s found that “reading just the signatures simplifies things very much.”

Adriaan allowed that the lower-effort path Eugene suggests is feasible, but added that there are long-term, bigger-pictures around parallel maintenance of Scala 2 and Scala 3 until Scala 2’s end of life, which won’t come until Scala 3 has been out for some time.

Martin questioned whether leaving out the method bodies would save that much effort, given that most of the differences between Scala 2 and 3 are in the type system, and all of the type system is present in signatures. Sébastien concurred; “it doesn’t matter” if the two compilers emit the same bodies. (Adriaan had mentioned this earlier as a possible goal, but not a necessary goal.)

Adriaan acknowledges that these questions aren’t settled yet, but he thinks that arriving at a final design through a process of investigation is covered under the proposal, the proposal doesn’t fix the final design.

Jon asked Eugene if a shared back end would benefit the rsc project. Eugene said maybe, it’s a longer term question. “When we get to that point, we will investigate.”

James asked Adriaan if he agreed with Eugene’s suggestion of doing a one-quarter trial first. Adriaan said yes, that’s reasonable. “It’s always good to keep re-evaluating.” But it interacts with the questions of hiring and training; if someone needs to come up to speed, it might (for example) take two quarters instead of one to get far enough to even evaluate.

Chris asked what the success conditions of the initial trial period would be. Sébastien said the initial focus would be on the shared subset of the two language versions, support for 2-only and 3-only features could come later, just supporting the shared subset would be useful already. “I would be happy if we get to that point,” Adriaan said; James and Sébastien agreed.

After some discussion, the board agreed to vote first on the 1-year version, and then if the vote failed, vote again on a 1-quarter version.

Voting: five in favor, one opposed (Twitter), no abstentions, three members not present. Since the first vote passed, there was no need for a second vote. The proposal is accepted.

Other business

Forum moderation

Two board members had asked Jon before the meeting about recent moderation incidents on official Scala forums, under the Scala Code of Conduct. Jon now raised the issue before the board.

Some of the ensuing discussion involved names of individuals and details of particular incidents. These minutes will stick to more general descriptions.

Shortly before the meeting, Sébastien posted the following public statement on the matter: https://contributors.scala-lang.org/t/looking-for-a-moderation-process-to-enforce-the-coc/2330 The statement briefly summarized one of the recent incidents, including noting that the temporary bans that were made were now lifted. It also proposed some changes to how moderation is handled and asked for community input.

There was discussion of whether sufficient evidence is being provided when warnings and bans are issued. In the recent cases, not all of the relevant evidence still exists. Sébastien’s statement addresses this by proposing better recordkeeping, and he repeated this in the meeting. We also discussed recent events until everyone felt they were clearer on what had happened.

Martin stated support for the moderators. “People take this on as a service to the community,” even though “it’s really difficult and unrewarding work.” Still, “if we can improve our handling we should do that.”

Funding

Jon reminded us there was discussion at the June meeting about possible changes to the Center’s funding structure, but that hasn’t moved forward since, presumably because of the change of directors.

Sébastien said this is still on his mind, and he has begun looking at what other open source foundations are doing. As we discussed last time, the Center does have two sponsorship tiers, but the lower tier doesn’t give much benefit to the company (no votes), and so far only one company (Oracle) has signed up.

It’s apparently common elsewhere to have multiple levels of sponsorship, typically depending on the company’s size, but with better benefits offered to smaller companies.

James asked why there’s been so little interest in the existing lower sponsorship tier. Why isn’t clear, but regardless, it indicates that a change is probably warranted. Perhaps, James suggested, the Center could approach some smaller companies directly and ask them what would interest them.

Community feedback

In this part of the meeting, Bill and Chris both made further comments about the moderation issues.

Bill stated that “you need moderation” in open source. Without it “a lot of damage” can result. But “when you give someone bad feedback, it’s important to have examples, so it’s good to keep track of that stuff.”

Chris noted that Typelevel has adopted the Scala Code of Conduct as well. Many Typelevel projects have changed or are considering changing from the old Typelevel code to the Scala code. Any improved handling of moderation is of interest to them as well.

Jon asked why change codes. Chris characterized the old code as “don’t do bad things”, while the new code is more about creating a positive environment.

Bill also relayed community support for attention to migration (with SCP-018 as an example of that). “The most important thing people deal with is versions and upgrading and migration. That’s been our experience on ScalaTest. Anything that can help us get from version to version of Scala” is very important.

Conclusion

Several board members were unable to be present at this meeting. Jon will try to schedule the next meeting farther in advance this time to prevent this from happening again.

The next meeting will be virtual and will likely be held in early December, even though that’s less than three months away, to avoid conflicting with holidays.