Hello list,
In the course of this year we have developed a new website to automatically suggest free software for PDF display. Now that we are preparing to deploy a test version I see however a lot of issues, that are still remaining, the removal of which I deem extremely problematic.
The aim of my concerns is less of a technical nature but more of an organisational one - I don't see us bringing up the workforce to keep the site running as fancy as we had originally planned.
So this is it! We do now need to decide how to go on, as we cannot go on the way we did.
Let me start with a synopsis of the development so far:
Léopold and I started laying down the structure of the new site about a year ago in October 2012. We wanted to base the architecture of the new website on the designs of the current site. Even though we quickly developed some reservations about webgen, the current build system, we decided to keep it for the redrafted site. Partly we did this so we could keep existing styles and translations, partly I fear, we didn't want to step on anyones toes by fundamentally altering the sites workings - after all, objectively webgen was doing it's job. We decided to augment the static site with some perl scripts which would recognise the visitors web browser by its user agent string and select an appropriate PDF reader for display. The selection should be based on the availability of the reader on the operating system platform and on its ability to integrate with the web browser. It came as no real surprise, that parsing the user agent strings of web browsers is impossible to do gracefully. However this was kind of in the requirement specification and after different ready made Perl modules proved useless for the task we started implementing our own recognition module, which is more up to date than the available solutions at the time and seems to deliver adequate results so far. By the time Léo ended his internship with FSFE and left the project leaving me as the sole responsible for the development.
Testing and bug squishing of the site proved tedious. Even weeks after I believed the site to be running smoothly Hannes discovered situations in which the software wouldn't deliver a page at all. Without having been able to reproduce the errors, I believe them to be fixed now (not that I didn't believe that before).
It now turns out, that Heiki experiences problems, building the site on the productive server. This is again a problem which I can neither reproduce nor fully understand, though I am confident that together we will be able to fix it quickly - at least as long as the webgen versions in Debian remain compatible. While we can probably handle the remaining technical issues at this point, the site currently lacks the input data (not so much the technology) to make a sensible software recommendation. We had a hand full of readers registered for testing purposes. The data describing platforms and browsers for those readers is incomplete however.
Now, actually this is where it gets worse. At some point we were made aware of a reader called PDF.JS - something which seemingly catched some attention over the last month. Aside from the fact, that the suitability of PDF.JS as a standalone offline reader can be disputed, yet many people expect us to recommend it as an online reader anyway, PDF.JS is basically operating system agnostic which makes it somewhat tricky to even represent in the data structures on which we base recommendations. Only this we could handle.
But the problems continue: to make the installation of a reader as easy as possible we usually want to link directly to an installer. But how do we do this on platforms, on which it is impractical, uncommon or even impossible to just install software via a web link? This is the case today with most mobile platforms and GNU/Linux distributions. I have included a special case to enable aptURL-Links which are as far as I know only reliable in Ubuntu and even there only in Firefox. On most other GNU Linux distributions it would be nonsensical to link to a software installer altogether. On Android in the default configuration users will not even be able to install a software package delivered via the web browser, even if the projects would bother to offer a prebuilt package outside the play store in the first place.
In the upcoming months and years we expect UbuntuPhone, FirefoxOS, and possibly SailfishOS and GnomeOS and who knows what else to hit the market on mobile and desktop devices. This will make the situation infinitely worse. If we didn't expect to handle the runtime architecture of PDF.JS, I don't even dare to take a guess what new PDFreaders we will see on those platforms, let alone how to trigger an installer for them. Who wants to predict what the user agent strings on the new platforms will look like?
As the situation is currently, we lack the workforce to suit our data sets to the present day situation. It will take many times as much work to keep up with the changes we expect to see in the near future. Chances are that we will have to make technical adaptions in addition, to handle new concepts in software behaviour that we haven't seen yet.
Apparently we are not the only ones linking to alternative PDF readers. I've seen some websites of German governmental organisations link to the Heise software directory[1] along with PDF downloads. Heise is a publishing house for a number of German computer magazines. Even their list of Free and Non-Free PDF readers, although excellently maintained by a group of full time journalists, offers only a flat view similar to the one we show on the current version of pdfreaders.org. [1]http://www.heise.de/download/office/pdf/viewer-50000505011/?f=5s
Until today I have spent about 130 hours of FSFE time on the reimplementation of PDFreaders.org, not counting the time Leo and some others invested. With all our dreams and ideas of what the site should ideally look like this could go on indefinitely. We have to cut our losses and draw a line here.
This means, we could trim back our original plans drastically and concentrate on suggesting only a hand full of desktop readers on a few common platforms. In particular we would neglect most Free Operating Systems and mobile platforms. Preferably we would drop the greeter page[1] and take the readers page[2] for the index, so that erroneous suggestions don't carry so much weight. [1] http://pdfreaders.plutonium.fsfeurope.org/index/index.en.html [2] http://pdfreaders.plutonium.fsfeurope.org/readers/index.en.html
Alternatively, we can stick with the current version of pdfreaders.org, maybe paint the site up a little, and see to it, that the table displays up-to-date readers with working links. If we find yet one other setup, for which the new pdfreaders test site displays only a blank page, I'm all for this one.
I believe for both solutions we should anticipate another 10 to 20 hours of work time, under the premise, that even a cut back version of the new site, should spend a month or so in a test deployment while the static version would require less time being checked out by volunteers.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
Hi Paul,
I'll try and share some constructive ideas and experience.
First, thank you for all your work on the project, from a mere observer (me).
Second, consider condensing your report / proposals into a shorter mail of about 2-3 paragraphs, with specific questions at the top. That'll make it easier to engage with for most people on this list and increase your chances of replies.
Third, my experience with Webgen on TDWYT led me to migrate it to Pelican. If you work on similar FSFE projects in future, I recommend moving away from Webgen for lots of reasons. Pelican and Jekyll are two decent alternatives.
Fourth, most of these problems could be greatly eased by sharing the maintenance and development burden with our extensive community. Specifically, I suggest hosting the code on a popular public code hosting platform and encouraging and monitoring pull requests.
I proposed to you once before a simple text-based way of storing PDF-reader information, such that non-technical people could add and update entries without having to touch code. I'm not sure how close we are to that already with the PDFR website? I, for instance. This is also the approach we're taking for both TDWYT and DFD. So far I've had 6 translations and 3 sticker designs from volunteers via git pull requests on Gitorious.
Fifth, I feel that the design of the site (CSS) is one of the most important parts, and deserves special attention. Unfortunately, PDFR's current appearance turns off some visitors, yet we currently have good volunteer designers around FSFE, and applying a new theme is the sort of high-profile task that may attract volunteers.
Finally, thanks again for bringing these problems to team. Bearing bad news is never fun.
Sam.
- -- Sam Tuke Campaign Manager Free Software Foundation Europe IM : samtuke@jabber.fsfe.org Latest UK Free Software news: uk.fsfe.org Is freedom important to you? Join the fellowship.fsfe.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Sam Tuke samtuke@fsfe.org, Mon, 23 Sep 2013 11:50:01 +0200:
First, thank you for all your work on the project, from a mere observer (me).
;-)
Second, consider condensing your report / proposals into a shorter mail of about 2-3 paragraphs, with specific questions at the top. That'll make it easier to engage with for most people on this list and increase your chances of replies.
Yeah short mails are great, and I tried to keep the paragraphs short and easy to read, but I felt it necessary to deliver some extended explanation.
Third, my experience with Webgen on TDWYT led me to migrate it to Pelican. If you work on similar FSFE projects in future, I recommend moving away from Webgen for lots of reasons. Pelican and Jekyll are two decent alternatives.
Very good to know, thank you!
Fourth, most of these problems could be greatly eased by sharing the maintenance and development burden with our extensive community.
I tried to do that, one of my past mails describes how to easily modify the reader information, unfortunately we received no community support then. Suggestions about what should be altered may be helpful in general but without modified reader files attached they increase my workload.
Specifically, I suggest hosting the code on a popular public code hosting platform and encouraging and monitoring pull requests.
Hmmm... using external services for code hosting goes a lot against my ideals. We can and we do provide the infrastructure for code hosting already. Different programmers massing up coding projects on very few websites leads to a centralisation of the internet in general, and I feel strongly that we should promote an alternative behaviour.
I proposed to you once before a simple text-based way of storing PDF-reader information, such that non-technical people could add and update entries without having to touch code. I'm not sure how close we are to that already with the PDFR website?
We are pretty close. The reader definitions are short JSON Files, one for each reader. I'd rather not put in the work for simplifying this even further until we know, that it will be worthwhile.
I, for instance. This is also the approach we're taking for both TDWYT and DFD. So far I've had 6 translations and 3 sticker designs from volunteers via git pull requests on Gitorious.
The repository can be browsed publicly via http and svn, though I admit I don't know how many people checked out the code. I received no mails with modified definition files after my request on the pdfreaders list, but at least Heiki contributed some work via svn.
Fifth, I feel that the design of the site (CSS) is one of the most important parts, and deserves special attention. Unfortunately, PDFR's current appearance turns off some visitors, yet we currently have good volunteer designers around FSFE, and applying a new theme is the sort of high-profile task that may attract volunteers.
Yep, as I wrote, we could pimp it up a little, no matter how we proceed.
Thank you for sharing your ideas. - -- Paul Hänsch █▉ Jabber: paul@jabber.fsfe.org Webmaster █▉█▉█▉ Support the FSFE Free Software Foundation Europe ▉▉ http://fsfe.org/support/?paul
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
On 28/09/13 19:00, Paul Hänsch wrote:
Specifically, I suggest hosting the code on a popular public code hosting platform and encouraging and monitoring pull requests.
Hmmm... using external services for code hosting goes a lot against my ideals. We can and we do provide the infrastructure for code hosting already. Different programmers massing up coding projects on very few websites leads to a centralisation of the internet in general, and I feel strongly that we should promote an alternative behaviour.
Yes, I see your point. I suppose in this case I value visibility of the code (and potential participation) above the problematic side effect of hosting somewhere centralised like Gitorious. This isn't the place to debate our views on that subject however, so let's park the issue for now.
I proposed to you once before a simple text-based way of storing PDF-reader information
We are pretty close. The reader definitions are short JSON Files
Glad to hear it! Makes sense.
Fifth, I feel that the design of the site (CSS) is one of the most important parts
Yep, as I wrote, we could pimp it up a little, no matter how we proceed.
Not sure how to proceed here; I'd love to work on the design (both concept and implementation) but I don't have time, unfortunately.
Matthias: what do you think about writing to designers@ asking them for assistance. If so, I recommend a very tight scope in the request.
Thank you for sharing your ideas.
My pleasure :)
Sam.
- -- Sam Tuke Campaign Manager Free Software Foundation Europe IM : samtuke@jabber.fsfe.org Latest UK Free Software news: uk.fsfe.org Is freedom important to you? Join the fellowship.fsfe.org
Hello,
2013/9/18 Paul Hänsch paul@fsfe.org:
But the problems continue: to make the installation of a reader as easy as possible we usually want to link directly to an installer. But how do we do this on platforms, on which it is impractical, uncommon or even impossible to just install software via a web link? This is the case today with most mobile platforms and GNU/Linux distributions. I have included a special case to enable aptURL-Links which are as far as I know only reliable in Ubuntu and even there only in Firefox. On most other GNU Linux distributions it would be nonsensical to link to a software installer altogether. On Android in the default configuration users will not even be able to install a software package delivered via the web browser, even if the projects would bother to offer a prebuilt package outside the play store in the first place.
Relax, I don't think you need to stress yourself with all possible options.
The main purpose of the site is to be an alternative to www.adobe.com/en/products/reader.html, and the mission of the site is to get public administration sites to link to pdfreaders.org instead of adobe.com. The goal is met if you can download installers for Windows and Mac. All the others are just extras, and if you don't feel good about making more download options, then don't. The site will be just fine.
Thanks for your work!
Otto Kekäläinen otto@fsfe.org, Tue, 24 Sep 2013 08:08:05 +0300:
2013/9/18 Paul Hänsch paul@fsfe.org:
...
Relax, I don't think you need to stress yourself with all possible options.
Am I right to understand, that you had a part in laying down the original specifications of the site? The oral description of the requirements, which I received about a year ago, seemed indeed to necessitate the steps described by me.
The main purpose of the site is to be an alternative to www.adobe.com/en/products/reader.html, and the mission of the site is to get public administration sites to link to pdfreaders.org instead of adobe.com. The goal is met if you can download installers for Windows and Mac.
Then at the current point in time this goal is not met.
All the others are just extras, and if you don't feel good about making more download options, then don't. The site will be just fine.
This assessment is in vast contrast to the feedback I received from Hannes and to my own evaluation.
The site you are speaking about is currently online on: http://pdfr.plutonium.fsfeurope.org Please make sure that we are talking about the same thing when you say that this is "just fine".
How to make it so, that it *will* be just fine, was the original question of my last email.
I realise, that I might sound pushy here, but the point I raised is an actual, specific, and immediate issue, which requires attention. It cannot just be waived away.
The question is still in the room: How should we continue? I've listed the available options.
Hello,
2013/9/28 Paul Hänsch paul@fsfe.org:
Otto Kekäläinen otto@fsfe.org, Tue, 24 Sep 2013 08:08:05 +0300:
2013/9/18 Paul Hänsch paul@fsfe.org:
...
Relax, I don't think you need to stress yourself with all possible options.
Am I right to understand, that you had a part in laying down the original specifications of the site? The oral description of the requirements, which I received about a year ago, seemed indeed to necessitate the steps described by me.
I have participated in a PDF readers workshop some years ago, but I haven't had any part in designing anything of the renewal you have been working on.
The main purpose of the site is to be an alternative to www.adobe.com/en/products/reader.html, and the mission of the site is to get public administration sites to link to pdfreaders.org instead of adobe.com. The goal is met if you can download installers for Windows and Mac.
Then at the current point in time this goal is not met.
? At least the current public page has columns for Windows and Mac OS X
All the others are just extras, and if you don't feel good about making more download options, then don't. The site will be just fine.
This assessment is in vast contrast to the feedback I received from Hannes and to my own evaluation.
The site you are speaking about is currently online on: http://pdfr.plutonium.fsfeurope.org Please make sure that we are talking about the same thing when you say that this is "just fine".
Well.. for example when I click on "show 11 other suggestions" the URL http://pdfr.plutonium.fsfeurope.org/readers/ opens but the page fails to load, it is just blank. This does not give a good impression, but otherwise the simplification of the new site compared to the old one is good.
In stead of just throwing around opinions this kind of project benefit of analysing the visitor statistics and analyzing how the PDF readers success rate develops (currently 26% of the reported sites have changed their link from adobe to pdfreaders). But unfortunately I don't have time to do it, but I hope the FSFE campaign people would take some time to look at what methodologies there are and try to measure the success of changes in campaigns to validate that each change indeed is for the better, or if not, then try multiple changes to find which had the best effect.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
On 28/09/13 18:48, Otto Kekäläinen wrote:
In stead of just throwing around opinions this kind of project benefit of analysing the visitor statistics and analyzing how the PDF readers success rate develops (currently 26% of the reported sites have changed their link from adobe to pdfreaders). But unfortunately I don't have time to do it, but I hope the FSFE campaign people would take some time to look at what methodologies there are and try to measure the success of changes in campaigns to validate that each change indeed is for the better, or if not, then try multiple changes to find which had the best effect.
It sounds like you're suggesting A/B testing [1]. That would indeed be the most meaningful way to test the changes and see what is most effective for visitors.
It does take time however, and there is currently no campaign team, budget, or time allowance for PDFR as I understand it. In fact the campaign is in negative budget as it's gone over it's allocated development time.
Matthias: can you give Paul some guidance, and Otto some idea if stats assessment or A/B testing will be possible?
Thanks,
Sam.
1. https://en.wikipedia.org/wiki/A/B_testing - -- Sam Tuke Campaign Manager Free Software Foundation Europe IM : samtuke@jabber.fsfe.org Latest UK Free Software news: uk.fsfe.org Is freedom important to you? Join the fellowship.fsfe.org
* Sam Tuke samtuke@fsfe.org [2013-10-07 11:41:39 +0200]:
Matthias: can you give Paul some guidance, and Otto some idea if stats assessment or A/B testing will be possible?
For the record: I will discuss the issues with Paul. The problem is that we have very little resources for the campaign, as it should have already been finished in February. Before we such kind of testing for pdfreaders.org I would like to invest more on fsfe.org, or other campaigns.
Regards, Matthias
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Hey,
On 18/09/13 17:51, Paul Hänsch wrote:
The aim of my concerns is less of a technical nature but more of an organisational one - I don't see us bringing up the workforce to keep the site running as fancy as we had originally planned.
So this is it! We do now need to decide how to go on, as we cannot go on the way we did.
While I agree we do not currently have enough volunteers to keep all of the information up-to-date, I am of the opinion that we should deploy the new site as planned and leave the decision of which fields in the JSON files to fill to the PDF readers team. Maintaining all the information we have fields for is not realistic at the moment, but it might be in the future. The new architecture actually makes the site easier to maintain than before. Especially adding and removing readers – yes, we do need to do that: some listed readers are unmaintained, and one reader contains non-free software – has become easier on the orders of magnitude.
It now turns out, that Heiki experiences problems, building the site on the productive server. This is again a problem which I can neither reproduce nor fully understand, though I am confident that together we will be able to fix it quickly - at least as long as the webgen versions in Debian remain compatible. While we can probably handle the remaining technical issues at this point, the site currently lacks the input data (not so much the technology) to make a sensible software recommendation. We had a hand full of readers registered for testing purposes. The data describing platforms and browsers for those readers is incomplete however.
The technical problems I have been experiencing are most likely the fact that the new version probably requires altering files outside the SVN repository (possibly updating webgen, installing Perl modules or alike), which I am unable to do and system hackers seem to have no time for. Other than that, I am fairly certain that the new site can be deployed to test.pdfreaders.org and pdfreaders.org fairly easily.
But the problems continue: to make the installation of a reader as easy as possible we usually want to link directly to an installer. But how do we do this on platforms, on which it is impractical, uncommon or even impossible to just install software via a web link? This is the case today with most mobile platforms and GNU/Linux distributions. I have included a special case to enable aptURL-Links which are as far as I know only reliable in Ubuntu and even there only in Firefox. On most other GNU Linux distributions it would be nonsensical to link to a software installer altogether. On Android in the default configuration users will not even be able to install a software package delivered via the web browser, even if the projects would bother to offer a prebuilt package outside the play store in the first place.
Where providing some data does not make much sense, we ought not to provide such data at all.
In the upcoming months and years we expect UbuntuPhone, FirefoxOS, and possibly SailfishOS and GnomeOS and who knows what else to hit the market on mobile and desktop devices. This will make the situation infinitely worse. If we didn't expect to handle the runtime architecture of PDF.JS, I don't even dare to take a guess what new PDFreaders we will see on those platforms, let alone how to trigger an installer for them. Who wants to predict what the user agent strings on the new platforms will look like?
I am of the opinion that we should deal with such issues if/when they arise.
Until today I have spent about 130 hours of FSFE time on the reimplementation of PDFreaders.org, not counting the time Leo and some others invested. With all our dreams and ideas of what the site should ideally look like this could go on indefinitely. We have to cut our losses and draw a line here.
Indeed. However, I think that we should simply go live with the new site, possibly making small updates to the JSON schema to make more of the data optional. E.g., if we cannot/do not want to maintain version information for the readers, we would be able to stop doing that. But other than making most reader data optional, I do not currently see a need for large modifications and insist on deploying the development version ASAP and just seeing how it goes.
If porting to one of the alternatives Sam mentioned can be easily done and would ease further maintenance and development, then that is also worth considering.
Personally I am of the opinion that the work that has went into the new site thus far has been of high quality and the new site, with all its flaws, is better than the old one. (Thanks, Paul!)
Cheers, - -- Heiki "Repentinus" Ojasild FSFE Fellowship Representative mailto:repentinus@fsfe.org xmpp:repentinus@jabber.fsfe.org http://blogs.fsfe.org/repentinus/
"Heiki "Repentinus" Ojasild" repentinus@fsfe.org, Thu, 26 Sep 2013 14:45:30 +0000:
While I agree we do not currently have enough volunteers to keep all of the information up-to-date, I am of the opinion that we should deploy the new site as planned and leave the decision of which fields in the JSON files to fill to the PDF readers team.
Deploying the new site "as planned" kind of involves keeping or a least having the information on it up to date.
Maintaining all the information we have fields for is not realistic at the moment,
agreed
but it might be in the future.
... or maybe soon after that.
The new architecture actually makes the site easier to maintain than before. Especially adding and removing readers – yes, we do need to do that: some listed readers are unmaintained, and one reader contains non-free software – has become easier on the orders of magnitude.
I'm happy to read that.
The technical problems I have been experiencing are most likely the fact that the new version probably requires altering files outside the SVN repository (possibly updating webgen, installing Perl modules or alike), which I am unable to do and system hackers seem to have no time for. Other than that, I am fairly certain that the new site can be deployed to test.pdfreaders.org and pdfreaders.org fairly easily.
Yes, the initial deployment might well require shell access to the server.
Where providing some data does not make much sense, we ought not to provide such data at all.
OK, that would be a standpoint we can elaborate on. So we accept that we have occasions where we omit a download link...
I am of the opinion that we should deal with such issues if/when they arise.
I am of the opinion that "we" should be exchanged for a more specific term. I understand that on many occasions, this kind of work has been seen to somehow. This time however pulling in active volunteers for the project seems to be an issue.
Indeed. However, I think that we should simply go live with the new site, possibly making small updates to the JSON schema to make more of the data optional. E.g., if we cannot/do not want to maintain version information for the readers, we would be able to stop doing that. But other than making most reader data optional, I do not currently see a need for large modifications and insist on deploying the development version ASAP and just seeing how it goes.
OK, that's pretty much what I proposed a while ago meeting some resistance then. Resistance, mind, which is well substantiated too.
If porting to one of the alternatives Sam mentioned can be easily done and would ease further maintenance and development, then that is also worth considering.
Sam proposed alternatives regarding the build-system and I too think, that we should keep them in mind. Though I believe this is not an immediate maintenance issue and in this case I am the one who believes that we can postpone the work until it becomes necessary. Regarding the storage of reader information we already use a simple text-based way. The format can probably be simplified further, but not easily and not without knowing what will become of the site now.
Personally I am of the opinion that the work that has went into the new site thus far has been of high quality and the new site, with all its flaws, is better than the old one. (Thanks, Paul!)
I sure feel flattered ;-)