Igalia’s Brian Kardell sat down with Neil Soiffer, long time MathML advocate and chair of the current MathML Refresh Community Group to chat about MathML, it’s history, challenges and what’s next.
Brian Kardell: Ok, so, I am Brian Kardell. I am from Igalia, where I am a developer advocate. We thought it would be fun and interesting for posterity to do some interviews around our MathML efforts. So, I’m here with Neil who is the chair of the MathML Refresh Community Group and we’re going to have a conversation about MathML. So Neil, I think a lot of people coming to MathML and hearing about it... a lot of them are probably new to it, and they probably don’t realize that conversations about Math on the web go way back. And, I know that you have been here for really a lot of it, can you tell us a little about that... your history, and the history of Math on the Web?
Neil Soiffer: Yeah, I was part of it from way back from the start... back in the 90's even. I worked for Wolfram Research and we were interested in promoting a web standard. There had been some initial work on an inital standard that hadn’t gone anywhere. It was very minimal. So, we proposed something to the W3C... and that we should start a group. They were very favorable to that idea so we convened a number of people. We had a few conferences and workshops together. It took a while, but in... 1998, I think it was, we became the second XML application out there on the web. So, it goes back quite a ways. So that was a long time ago, and I was part of that, and I've been part of the community working on MathML ever since. Which makes me feel kind of old.
Brian Kardell: (chuckles) Yeah. I think a lot of web standards have a long history, but the late 90’s, that really puts it into perspective. I guess it’s historically been kind of a roller coaster ride?
Neil Soiffer: (slight sigh) Yeah. There have been all of these periods of enthusiasm, and then that gets damped down. And then it comes back up again. You know, we had this proposal, it was 1998, and that was great. It was a compromise, as most standards are, and it was huge because of that compromise. That was maybe a problem because that made it harder to implement. Another thing was, as I said, it was an XML application — so we had hitched our wagon to that, but that did not turn out to be the direction the web went in. On the other hand, so did SVG, and SVG has had a little more buy-in on the web than MathML has.
Brian Kardell: Yeah, I’m glad you mention SVG, because I think there are two really interesting things about this. One is that both MathML and SVG are specifically integrated into the HTML parser later on and so they’re “special” in history. But also, like with SVG, because of that aspect of it — MathML has this huge ecosystem around it outside of browsers... From things to understand and transcribe it from handwriting analysis and pictures and just, all this cool stuff… But browsers, I think, have been really where the rollercoaster ride has been, right?
Neil Soiffer: That’s right, yeah. It’s been a little strange that all of the browser implementation work has been done by volunteers — it was never initially picked up by the browser vendors themselves. I’m not sure why. I could guess that maybe there is some Math phobia — or maybe math wasn’t the sexy thing to work on or something… I don’t think anyone ever said “gee this is a terrible standard and we don’t want to do it” it just never seemed to become a priority for them. That’s been a disappointment. We’ve seen ups and downs, and I can get into that, but yeah — it’s used in so many places as you indicated. It’s used a lot in publishing because they really do like XML workflows in publishing. But, on the web, it just hasn’t been able to get into all the browsers in the same way other things have. It has been supported in several browsers over the years, but yeah… right now, it is making a comeback — and the roller coaster is on a big upswing, I think.
Brian Kardell: Yeah… Sort of my favorite bit of history here is that MathML has been really the victim of just so much unfortunate timing. So many things that have little to do with MathML itself, but about changes. For example, which browsers were popular at the time, which ones you had to get into — and then also, as you said, the shift from XML and XHTML looking like they are going to be the future and then it leaves W3C and goes into WHATWG as HTML, eventually integrated into the parser. There’s just a really bunch of interesting things there about the unfortunate timing.
Neil Soiffer: Yeah, well, you’ve jumped through probably two decades worth of time there, but … yeah. Initially, way back around 2000, it’s probably hard for people to remember even, but the browser back then was IE. Maybe it was IE6 at that point — and — Design Science, a company I eventually joined, produced a plugin for IE called MathPlayer. It supported MathML, it was great. Firefox was a really small percentage, but it was a percentage and people wanted cross platform support. They got MathML support, but when they did it was for XHTML documents only. I can’t recall the reasons why, but, as you said, that seemed to be the trend back then that maybe XML was the way things would go. Turned out, not at all. This created this incompatibility because IE then did not support XHTML. So, we ended up hacking the IE plugin so that it would transform XHTML to HTML, and that was all very good. We were all very positive at Design Science that we had solved this problem — because, now all you had to do is put out XHTML documents. But… there wasn’t an easy way to do that. Most people were putting out HTML. So there was confusion about this, and this lasted until about 2010, I guess, when Firefox finally supported it as HTML. But, by that point IE was falling away, and they had stopped supporting the interface that MathPlayer needed in order to work — and now Chrome was getting to be really popular. So, a volunteer worked and, had some pretty good success, getting an implementation into Chrome [through webkit]. And so, at that point — I think we were up to 4 browsers supporting MathML in some form or other, and things were looking positive, and I think there was a lot of excitement that Chrome was finally going to get it — it was the dominant browser. But, unfortunately when there was a fork to blink off of it, they yanked it all out. There were issues, maybe with security. But in the end, that was a kind of a downside turn for MathML in browsers because this dominant browser no longer supported it. The support in IE had kind of failed as IE went away. It was a bleaker period for MathML support. There were a lot of people who were very upset about Chrome support being yanked. It is ranked as the third, or maybe even the second issue that is sitting there in the Chrome logs as people wanting it back. There definitely were people saying great things about MathML, but it just wasn’t clear why it wasn’t getting the attention from implementers.
Brian Kardell: There was a little bit of a double whammy here, actually, as well. Right about at, approximately at the same time that the math support was removed… and… oh… actually, when we say the math support was removed, we should say math rendering support, more correctly because the parser in all of the browsers still parses MathML specially because…
Neil Soiffer: Yeah, that’s something that I think is really important. MathML did go into HTML5, but HTML5 basically defined how the parser was going to turn it into the DOM — it didn’t say anything about how you must render this.
Brian Kardell: Right. It just delegated that support to supporting the rendering of embedded MathML. But, the other sort of double whammy… or, maybe this is the triple whammy, or quadruple whammy? But, it’s right about the same time that support was removed — or — maybe just before, a thing that I was involved in — the Extensible Web Manifesto was published. And, the thing is — it argues how we move forward, and it distinguishes between existing high level features that make up the platform, and new features. And, I think this actually played out pretty badly for MathML too because, like, which one is it? Because we lost support…
Neil Soiffer: Yeah, I agree, you are kind of stuck in this limbo where you’re not really supposed to add it because it’s in there, and yet, it’s not supported, so you need to add it on. It’s a catch 22.
Brian Kardell: Yeah, it created a difficult situation because of the split with blink because not just chrome, but opera followed — and so in a very short period of time — like, maybe a week — the trend for MathML seemed to turn increasingly bleak…. But… well… here you are again, with us, and the MathML Core specification, in the MathML Refresh Community Group, chairing it. And, well, I just want to give you the opportunity to answer this, because I know a whole lot of people have asked me as I talk to them about MathML “…ummm…well… [sigh]… is it going to be any different this time”?
Neil Soiffer: Yeah, sure. Well, as bleak as it looked in the browsers, it actually had a lot of buyin from publishers and in the accessibility community because that is what the screen readers could use. That is what the dyslexia tools, which wanted to do highlighting of math — that’s what they interpreted. So, there was a lot of non-browser based support for MathML out there — and that support all still exists, and there are still a lot of people who really want to see MathML supported natively. So, while MathML itself wasn’t natively supported in all of the browsers, there is a wonderful — and very large — library called MathJAX. It’s a JavaScript library that does support MathML. It also supports TeX. So, there’s a lot of Math out there and the nice thing about MathJAX is that it can hide the MathML (since it can’t be displayed in Chrome) but they hide it in a way that screen readers can still see it. Wikipedia does the same thing, Khan Academy did the same thing, and, in fact… I was just helping my son with a calculus problem and needed to refresh my memory about how something worked, and I was looking at a bunch of web pages, and they almost all used MathJAX or hid it the same way because they did some server rendering — but they’re all accessible because they have the MathML. So, it’s there, it’s just not being layed out in the browsers. And, as wonderful as the MathJAX library is, its a JavaScript library — and there are just problems with JavaScript libraries beyond just load time, interacting with other things. So, I’m maybe losing the initial train of thought, but I think that this time around, the Igalia work is pushing it foward so that the rendering part of it can show up, and the MathML that’s been there all along can just be put in to be rendered. So, the support that Igalia has given to the implementation and the support that the Chrome team has given Igalia, all makes me very excited about the future. Oh, and also the work the Igalia team has done to help make the other browser engines consistent with this newer work that the refresh community is doing. It just has all of the signs that I think this time, we really have it.
Brian Kardell: I agree with…I agree with the .. the idea that this time is gonna be different. Because, not just we have the support and the implementers and the sort of good working relationship — but because the MathML Refresh Community Group, which you chair, took the time and actually answered all of the questions and concern that were lacking. Because of this weird history, there was this big gap. MathML, even in the browsers where it was implemented, it was kind of… undefined how it would work in the browsers.
Neil Soiffer: Yeah… a little defense of MathML, but also, a strong defense of what the group is doing now: Way back in 1998, when people wrote specs, the idea was that browsers would do the innovation and they didn’t want specs that forced everybody to do exactly the same thing because there was a lot of innovation coming on. So if you put out a web page, it wouldn’t on the same on one browser… exactly the same, on another. And that was, I think, a goal back then because people wanted to see — you know — what’s the best way to do things. And, that’s changed dramatically. Now we think “if I put out a web page it should look the same everywhere” and the goal of the refresh group is to do that same thing for MathML. So, one of the things we’re doing in MathML Core work, the part that goes into a browser, is we are looking at how MathML integrates with CSS, and defining layout in terms of the CSS Layout Engines so that the MathML will layout identically in every browser. We’re paring out the parts that are problematic or aren’t used very much, letting those be handled for now by a polyfill to maintain backward compatibility. The issue that made it difficult to implement MathML in a browser, we’ve looked at those and we’ve tried to make that not a problem, and get everything to be identical everywhere so that MathMl fits the way the Web works nowadays.
Brian Kardell: Yeah, definitely. And also, I wanted to make sure you understood the context of my point, because it was not in any way to say something negative about MathML. It’s that it comes from a co-evolutionary past. It was evolving at the same time as HTML and the DOM and how we think about and write up standards. Like, do we write them up in this way where there is infinite room for browsers to interpret them in many ways? Like, do we write the words first without worrying about the implementability of it. Because of this, the spec was not very inline with today’s web platform, and I think that the work that the MathML Refresh CG group has done to integrate it with CSS and the way that we think about units and the roles of attributes and DOM nodes and the DOM itself, actually, is my favorite one… And how we look forward to the future together and integrate with it all together. In fact, one of the things that makes me very very positive is, as you say we have had some positive support from Google and patches have begun landing upstream in the past couple of weeks.
Neil Soiffer: Yeah, one of the things that I’m very enthusiastic about is that years ago, when the volunteer that I mentioned trie to put MathML into Chrome — he told me that it was like pulling teeth to get anyone to do a review of his code. And now, these pathces that are landing, I’ve seen very quick response — people are very excited to see these come in. They have comments about how to do some things better, but the interest is there, and I think that’s a world of difference from before and that makes me very, very this time around.
Brian Kardell: So… Another thing that I wanted to ask about — and you have actually answered it a little bit already… When I talk to people about this, they point to — as you already said — the fact that there is a lot of MathML on the web today — or — there is a lot of math on the web today, even more… A superset of MathML I guess. So, it is there. It exists. You can see it. So, kind of, what’s the problem? Why does it really matter to have native support — instead of just some amazing tooling that can turn it into SVGs or images or something like that?
Neil Soiffer: Yeah. So.. Again, it comes down to this issue of standards. Once people have standards, they can innovate more because they don’t have to reinvent the wheel every time. There are a number of math editors out there that have their own implementations to draw the math. Except for the very big ones like MathJAX, they tend to have a number of rendering issues when you start stretching things like parenthesis, you might see gaps in them, or they might not stretch very well at all. They may look really funny. And, there are other issue — like if you want to start building interaction, you don’t have a standard underneath to draw — what one person uses underneath can’t be used by someone else, because they render it a little differently using divs and spans and playing different tricks. Once you have that standardization, you have, essentially the ability to stand on the shoulders of others and… stuff happens much better. So, whereas you might see some very nice work — like: Desmos has some very nice interactive stuff — it’s just Desmos only. I can’t export it for someone else and do much of anything other that they support export and import from MathML, but I’m not able to use their interactivity in any particular way. So, I see the standardization as allowing all sorts of innovation to really take off as opposed to being one shot efforts that can’t integrate with each other and can’t build on each other.
Brian Kardell: I think there’s also a really strong case to be made here that math is text, and text is sort of the web’s thing and putting text into images and things is not..Well, it seems very weird trying to think about how you manipulate the rendering of text itself with JavaScript without some other like really core fundamental primitives that are currently not there. It feels bad, right? And it feels worse to me even that this is sort of almost discriminates against this one particular kind of text which is fundamentally important to STEM the origins of the web and I think, society. I think there’s sort of a like a [argument about] what should the web be and do, for I don’t know, ethical reasons — I suppose is maybe the thing I’m trying to say here what do you think?
Neil Soiffer: Yeah. I mean, you know there’s… you could do look at all these studies that talk about STEM being important for people to get good jobs to understand the world better . There’s, you know… There’s lots of things about STEM and when it becomes harder for education to put out math on the web that’s just a blockage for people to learn it and become more proficient in it and it does seem like a disservice to a core part of, I mean not — you know not just math but it’s it’s science and engineering and medicine and all sorts of areas that it’s just harder to collaborate in because there’s no easy way to put the math out. Or, you know, there’s no way to build upon these things. So, yeah, it it’s always bothered me. I’ve had a phrase that every kid and from kindergarten through 12th grade has to take a math class and whereas everybody uses the web these days for regular text. math has been a kind of backward area. Maybe four years ago now Wikipedia finally buried the MathML inside of the math that it generated [for rendering] so it was at least except accessible but it’s still not as rendered as nicely as the regular text in Wikipedia and there’s just no — you know — rationale for why math should be held back that way.
Brian Kardell: Yeah I think I think basically we make it hard, and we make you think about… well, make you think about things that you don’t generally have to think about for text like flash of unstyled content, like dragging down the performance of your page, like potential failures in which you get something that is like — gibberish on your screen. So I think that is really interesting. You hit on, in your earlier statement — just now and a little bit before too, about how a standard lets you stand atop that and do interesting new things.. So, I’m curious – like do you have any particular hopes for what kinds of things a universal standard highly interoperable math on the web would have
Neil Soiffer: I think there’s a lot of possibilities for interactivity. So, people have written some editors for the most part they’re not as powerful as they could be if they didn’t have to spend so much effort trying to figure out how to get the rendering done. I think they could spend more effort on the interactivity. There are people that have done, you know, great things. I mentioned desmos. There were nice graphics nice interactivity and stuff but it’s it’s a world unto itself because they’re not building upon each others back, and so that desmos application: I can’t just pop it in to something else it’s it’s there it’s by itself and that’s the way it is and that’s unfortunate. There’s a ecosystem called Jupiter notebooks which is a sort of Python an in the web and that has become very popular. math I think still remains a bit of a problem there because again MathML’s not really there. The nice thing about Jupiter notebooks is I have the data there. I have programmability. I can produce graphs. I can peruse tables. I can manipulate the data in various ways and send it off to my colleagues and they look at it and they have that data there and they can do their own thing. But you can’t do that really with the math because if I ever wanted to do something with the math — I’ve kind of got to pull it out to its own little application, because it’s not a standard that’s really supported everywhere. So it’s to to look at some of these isolated things and say yeah well if I could just put them all together then I have this really neat little thing that I could start doing. And maybe I could, you know, take my handwriting and pump it over here and I don’t have to type and do that. And then I take it out there and I manipulate it again by doing a little bit more handwriting and crossing things out and so forth as I would do on a pencil and paper. You just can’t do that where everybody has to implement their own version of how to lay the math out and there’s just no standard that people can depend upon that works across all the browsers. So, there’s you know, once you start thinking about the interactivity you can start thinking about what about , in education, what kind of neat interactivity things can I do and there. There are some specialized programs that use their own specialized tools to do that but again they don’t build upon each other and it’s it’s such a sad state that everybody has to go and reinvent the wheel every time they want to do something with math. So, I’m very excited at once MathML support gets in there we’re gonna see all sorts of really interesting applications come up because people will be able to take advantage of what’s there and what clever things one person has done and incorporate that in math to the work that they’re doing, and not have to reinvent that part of it.
Brian Kardell: Yeah I think actually, for me, one of the things that’s really interesting about the things that you said there is MathJAX itself. Because… I think people hear us talk about wanting a native standard and why maybe MathJAX isn’t like the right final answer for this problem and they get the impression that we think MathJAX is good.. or, I mean, is bad. Personally, I think MathJAX is good because it’s enabled all these things…
Neil Soiffer: Yeah, I… You know: I want to I want to say absolutely I think MathJAX is wonderful it allowed math to show up on the web and the people that have done it have done a great job with the rendering and I am 100% behind them. It’s just they had to spend a lot of effort on things that they really didn’t shouldn’t have had to spend.Brian yeah, yeah — no this this is exactly going to the point that you were saying that I was trying to pull back into which was: MathJAX actually does like… they have explored considerably more than simply the rendering part, right? Yeah and, so really, I think a lot of the work that we’ve done in MathML core is to help enable them to continue to do that — but just not worry about the rendering. Because, there seems like a lot of the people who were involved with mathjax are now working with, like my coworker Joanie — the editor of ARIA, with you. Like, how can we make MathML exploration interactive and like, more accessible and explorable? And, I think there’s like lots of great things that they can do if they can stand on top of a standard where a whole bunch of the problems that a lot of us see with the current state of things are just resolved, right? Like that the library gets smaller, and you don’t have these like worries about like: When does your page render text and reflow and how much JavaScript you have to load to just see the thing? I just think all that gets a lot better if we give them a really good foundational core on which to buildYeah, I mean one of the things I mentioned I mentioned before as I’ve been involved in accessibility and accessibility is a really big deal using math and now and way back in mathplayer I added the ability to synchronize the speech and highlighting because for people with dyslexia it’s been shown that if you highlight a word it helps focus them as you speak it so that they can read the math better and there’s studies that show that actually math reading is twice as hard or they make twice the number of errors that they do in reading regular text and without a standard for how do I pick out the part that I’m going to highlight that synchronized highlighting has kind of been lost for the last decade because math player’s no longer doing the rendering. I think once MathML gets to be back as the standard way of displaying things we’ll get back to being able is for it something as basic as highlighting the math as it’s been just as it’s done for text now by the various tools that help people with dyslexia.
Brian Kardell: Yeah I think I think all this is really really great I think one of the things that’s like… probably hard for a lot of people to appreciate - it certainly was difficult for me to appreciate before getting involved with standards — and even the MathML work, is just how difficult text rendering actually is. And to do it in line with the browser in the way that the browser thinks about it and flows it and when it processes and participate in all of the rules and everything… I was talking to a friend here in Pittsburgh who is very well-known for design and the two of us were just thinking about how recently it seems we’ve gotten a lot more information about just how difficult the rendering of text is, and how magical we’ve made it. And also how that that is such a nice thing — that you don’t have to think about just how much complexity is there yeah.
Neil Soiffer: I mean, I completely agree. It’s taken for granted I think. Most people think you just are, like, well I guess just laying out some text maybe. There’s bold maybe, there’s italic, it’s just gonna wrap around the lines… I mean how hard is this? But in fact making it look nice and, you know, at the very lowest level having a nice looking font that’s gonna render cleanly on various displays of different resolutions and be clear… that’s incredibly hard stuff. And, to me, the layout of math is so much easier than that. People have put in so much work to make text look nice on a screen [for non-math] — it just pails in the amount of time the in on doing that then what it’s taken to do the math stuff and we all take the text for granted. I think people think “Oh laying out the math is so hard” but once you have a font that you know can do the type of things math needs to do and that includes being able to stretch things like parentheses and other characters — arrows and whatnot — once you have that the math layout is relatively quite simple to sit on top of all that work that other people who have done.
Brian Kardell: Yeah I’m I’m really excited to see to see all this stuff land and to also — like, imagine the larger impact on society in like a long time. I think that this has like good good potential for a lot of positive things like you mentioned Jupiter notebooks and things like that that are exploring I think it would be fantastic if… well not “it would be” — it will be fantastic when the web is able to finally meet — 30 years since its inception at CERN, be able to really meet the use cases that they had at CERN at the time.
Neil Soiffer: Yeah you know when you think back to where it all started as you said back at CERN and you know this is a scientific thing. Tim Berners-Lee was a physicist he wanted to put out his stuff and that still remains a little bit problematic how he can do that, and that’s a little strange. But, I’m really really excited that that problem after, as you said 30 years, is going away - and to see to make it so easy that people can do their research. It’s a great thing and I’m really happy to be part of pushing that forward and I’m really glad you’re part of that effort too because you’ve great help also.
Brian Kardell: Thank you and thanks for spending the time to talk to me. I think this was a fun chat that we had and hopefully people find it interesting.
Neil Soiffer: Well thanks for inviting me to do the chat hopefully some people learned a little bit about the history of MathML and why it’s gone up and down in the past and why this time it’s finally going to catch on
Brian Kardell
Neil Soiffer