first draft of an RMarkdown export plugin

[meik michalke]

hi,

here's what i've done so far:
https://galactica.psycho.hhu.de/seafile/f/37d018e8bb554c3385b2/
(URL is valid for seven days)

it's an rkwarddev script, and you'll need the latest development snapshot from
GitHub because i found & fixed some issues along the way:

library(devtools)
install_github("rkward-community/rkwarddev", ref="develop")

then just run the script, the plugin should be up and running once it's done.

i'm open to suggestions what's missing most or should be done totally
different.


viele grüße :: m.eik

[Thomas Friedrichsmeier]

Hi,

great!

On Fri, 12 Oct 2018 21:33:53 +0200

Post by meik michalke
i'm open to suggestions what's missing most or should be done totally
different.

First impressions:
- Unless a lot more options get added, I'd suggest to merge the two
tabs of the plugin.
- But perhaps you intend to add options for style and colors, for
instance?
- It would be cool (if possible) if the "Save as"-control was pre-filled
with the input filename + appropriate filename extension. That would
make it so much easier to use in many simple cases.
- Maybe a bit out-of-scope for the time being, but as an idea: Offer to
render a PDF to a temporary file and open it in okular. That would
essentially make it a print (preview) feature.

Regards
Thomas

[Stefan Rödiger]

Hi,

indeed a great thing! All tests worked well here.

A general remark on RMarkdon in RKWard. Methinks, RMarkdown is more like a method/philosophy than just a functionality. Rmarkdown is in data science with R very intensively used.
Why not put it as separate menu entry on a prominent place? Right now there is great functionality (the preview & and plugin) and more seems to come.

Regards
Stefan

Post by Thomas Friedrichsmeier
Hi,
great!
On Fri, 12 Oct 2018 21:33:53 +0200

Post by meik michalke
i'm open to suggestions what's missing most or should be done totally
different.

- Unless a lot more options get added, I'd suggest to merge the two
tabs of the plugin.
- But perhaps you intend to add options for style and colors, for
instance?
- It would be cool (if possible) if the "Save as"-control was pre-filled
with the input filename + appropriate filename extension. That would
make it so much easier to use in many simple cases.
- Maybe a bit out-of-scope for the time being, but as an idea: Offer to
render a PDF to a temporary file and open it in okular. That would
essentially make it a print (preview) feature.
Regards
Thomas

[meik michalke]

hi,

Post by Stefan Rödiger
indeed a great thing! All tests worked well here.

that's great to know!

Post by Stefan Rödiger
A general remark on RMarkdon in RKWard. Methinks, RMarkdown is more like a
method/philosophy than just a functionality. Rmarkdown is in data science
with R very intensively used. Why not put it as separate menu entry on a
prominent place? Right now there is great functionality (the preview & and
plugin) and more seems to come.

interesting thought. but doesn't it make more sense to have the export menu
for RMarkdown files where all other export options usually are? i.e., i didn't
even tell you where to look but it seems you were able to find and use it.

i can however see that it can be helpful to have a special context menu if the
currently active editor window is markdown, including a direct launcher for
the export plugin. so i wouldn't add a new permanent menu category, but rather
let RKWard detect the file type and produce a context sensitive submenu like
it already does for R code, help pages or data frames.


viele grüße :: m.eik

[Stefan Rödiger]

Hi,
actually I had to look a bit found it because I didn't find it. Then I read the source code and finally 'found' it.
Maybe others may have also some thoughts about that.

''special context menu'' is certainly the best practise IMHO. This would keep the interface lean and elegant.

BTW, thanks Thomas and Meik that you are working on this! Highly appreciated!

Kind regards
Stefan

Post by meik michalke
hi,

Post by Stefan Rödiger
indeed a great thing! All tests worked well here.

that's great to know!

Post by Stefan Rödiger
A general remark on RMarkdon in RKWard. Methinks, RMarkdown is more

like a

Post by Stefan Rödiger
method/philosophy than just a functionality. Rmarkdown is in data

science

Post by Stefan Rödiger
with R very intensively used. Why not put it as separate menu entry

on a

Post by Stefan Rödiger
prominent place? Right now there is great functionality (the preview

& and

Post by Stefan Rödiger
plugin) and more seems to come.

interesting thought. but doesn't it make more sense to have the export menu
for RMarkdown files where all other export options usually are? i.e., i didn't
even tell you where to look but it seems you were able to find and use it.
i can however see that it can be helpful to have a special context menu if the
currently active editor window is markdown, including a direct launcher for
the export plugin. so i wouldn't add a new permanent menu category, but rather
let RKWard detect the file type and produce a context sensitive submenu like
it already does for R code, help pages or data frames.
viele grÌße :: m.eik
--
dipl. psych. meik michalke
institut f"ur experimentelle psychologie
abt. f"ur diagnostik und differentielle psychologie
heinrich-heine-universit"at d-40204 d"usseldorf

[meik michalke]

hi,

Post by Thomas Friedrichsmeier
- Unless a lot more options get added, I'd suggest to merge the two
tabs of the plugin.

yes, i thought about it. i mainly hid the version selection in another tab to
not confuse people with a choice they probably don't fully understand. support
for v1 is there for backwards compatibility only, it looked like "too
important" if it was on the same tab. does that make sense?

Post by Thomas Friedrichsmeier
- But perhaps you intend to add options for style and colors, for
instance?

adding more options would either mean tinkering with the document itself
somehow, or probably use the configuration parameter of pandoc(), i.e.
generating a temporary configuration file.

it looks to me like pandoc() was designed to force you to make stylistic
adjustments in the .Rmd file directly, rather than choosing options. [but i'm
currently totally failing when trying to make an .md file use a different
LaTeX template, but that's not an issue with our implementation...]

Post by Thomas Friedrichsmeier
- It would be cool (if possible) if the "Save as"-control was pre-filled
with the input filename + appropriate filename extension. That would
make it so much easier to use in many simple cases.

absolutely, that's on my TODO list.

the browser element currently has it's drawbacks for use as a "save file"
option. i was missing an "overwrite" checkbox for existing files, and it would
have helped to append file extensions automatically (although that can be kind
of worked around by pre-filling the field as suggested).

also, even though the element has file extensions set for the input script, it
doesn't apply them to files that were automatically added because the plugin
was launched with the file currently opened. i'm sure this can be dealt with
with JavaScript logic, but i think this actually belongs to the element
features itself -- it should be marked red when the defined file extensions
don't fit the given file, and the submit button deactivated.

Post by Thomas Friedrichsmeier
- Maybe a bit out-of-scope for the time being, but as an idea: Offer to
render a PDF to a temporary file and open it in okular. That would
essentially make it a print (preview) feature.

ah, nice idea. will try.


viele grüße :: m.eik

[Thomas Friedrichsmeier]

Hi,

On Sat, 13 Oct 2018 21:01:45 +0200

Am Samstag, 13. Oktober 2018, 13:48:48 CEST schrieb Thomas

Post by Thomas Friedrichsmeier
- Unless a lot more options get added, I'd suggest to merge the two
tabs of the plugin.

yes, i thought about it. i mainly hid the version selection in
another tab to not confuse people with a choice they probably don't
fully understand. support for v1 is there for backwards compatibility
only, it looked like "too important" if it was on the same tab. does
that make sense?

I had guessed as much. Well, let's leave it like this for now. Do add a
<stretch> on the second tab, though, to fix the layout.

Post by Thomas Friedrichsmeier
- But perhaps you intend to add options for style and colors, for
instance?

adding more options would either mean tinkering with the document
itself somehow, or probably use the configuration parameter of
pandoc(), i.e. generating a temporary configuration file.

As far as I understand, you can override those using the output_format
parameter in render(). Of course that may be more work than it's worth.
I was just wondering what further options the plugin _might_ get,
eventually.

the browser element currently has it's drawbacks for use as a "save
file" option. i was missing an "overwrite" checkbox for existing
files, and it would have helped to append file extensions
automatically (although that can be kind of worked around by
pre-filling the field as suggested).
also, even though the element has file extensions set for the input
script, it doesn't apply them to files that were automatically added
because the plugin was launched with the file currently opened. i'm
sure this can be dealt with with JavaScript logic, but i think this
actually belongs to the element features itself -- it should be
marked red when the defined file extensions don't fit the given file,
and the submit button deactivated.

I'll have to look into those.

Regards
Thomas

[meik michalke]

hi,

Post by Thomas Friedrichsmeier
On Sat, 13 Oct 2018 21:01:45 +0200

support for v1 is there for backwards compatibility only, it looked like
"too important" if it was on the same tab. does that make sense?

I had guessed as much. Well, let's leave it like this for now. Do add a
<stretch> on the second tab, though, to fix the layout.

done.

Post by Thomas Friedrichsmeier

adding more options would either mean tinkering with the document
itself somehow, or probably use the configuration parameter of
pandoc(), i.e. generating a temporary configuration file.

As far as I understand, you can override those using the output_format
parameter in render(). Of course that may be more work than it's worth.
I was just wondering what further options the plugin _might_ get,
eventually.

very good hint -- i've re-written half of the JavaScript sections to now use
rmarkdown::render() in favor of plain knitr::pandoc() where it offers output
format objects, and that also did the trick with specifying templates!

check out the new version of the plugin, it's pretty great already:
https://galactica.psycho.hhu.de/seafile/f/37d018e8bb554c3385b2/
- now uses rmarkdown::render() most of the time
- templates and other in-doc configuration works well
- saves the target file without intermediate template file
- target is suggested automatically from source file name
- but prevents overwriting the source.md with the exported.md where needed
- some additional formats (like html_vignette or beamer presentation)
- i've also added both "first format" and "all formats" features of
render() to not be limited by the menu; in fact, i think the first is a
better default than picking one of the formats statically


viele grüße :: m.eik

[meik michalke]

Post by meik michalke
https://galactica.psycho.hhu.de/seafile/f/37d018e8bb554c3385b2/

[...]

Post by meik michalke
- i've also added both "first format" and "all formats"
features of render() to not be limited by the menu; in fact, i think the
first is a better default than picking one of the formats statically

i've now also replaced output_file with output_dir for those cases. but
there's one issue i can't resolve:

when the plugin starts with one of those two options and targenFile is not set
yet, you can't submit although i believe i've defined the "required" rules
accordingly. notably, you only have to scroll to any other format option and
back again to make the submit button available. how can i fix that?


viele grüße :: m.eik

[meik michalke]

hi,

some further tweaks to the export plugin, most significantly it exports HTML
vignettes properly:
https://galactica.psycho.hhu.de/seafile/f/37d018e8bb554c3385b2/
you should also upgrade XiMpLe and then rkwarddev one more time to the most
recent develop branch:

library(devtools)
install_github("rkward-community/XiMpLe", ref="develop")
install_github("rkward-community/rkwarddev", ref="develop")

(i fixed an issue in XiMpLe autoreplacing "&", "<" and ">" with HTML entities
in the generated XML code, which rendered embedded JavaScript useless if those
symbols are used. code in <![CDATA[ ]]> nodes is now save.)

btw: i noticed that the preview of vignettes produces a fine HTML document,
but RKWard doesn't really render it as a vignette. as far as i know, by
convention vignettes should be configured like

output:
rmarkdown::html_vignette

in the YAML header, and render() should also use "rmarkdown::html_vignette" as
output format. if the two don't match, all further configuration in that
context is skipped, like the presence of a TOC. so the preview can be very
confusing, if you correctly defined a TOC but it just won't show. could you
parse the header between the two "---" for the presence of
"[[:space:]]*rmarkdown::html_vignette" and adjust the rendering accordingly?

Post by meik michalke
i've now also replaced output_file with output_dir for those cases. but
when the plugin starts with one of those two options and targenFile is not
set yet, you can't submit although i believe i've defined the "required"
rules accordingly. notably, you only have to scroll to any other format
option and back again to make the submit button available. how can i fix
that?

viele grüße :: m.eik

[Thomas Friedrichsmeier]

Hi,

On Mon, 15 Oct 2018 00:55:14 +0200

Post by meik michalke

Post by meik michalke
https://galactica.psycho.hhu.de/seafile/f/37d018e8bb554c3385b2/

[...]

Post by meik michalke
- i've also added both "first format" and "all formats"
features of render() to not be limited by the menu; in fact, i
think the first is a better default than picking one of the formats
statically

i've now also replaced output_file with output_dir for those cases.
when the plugin starts with one of those two options and targenFile
is not set yet, you can't submit although i believe i've defined the
"required" rules accordingly. notably, you only have to scroll to any
other format option and back again to make the submit button
available. how can i fix that?

bug. Now fixed in git.

Regards
Thomas

[meik michalke]

hi,

Post by Thomas Friedrichsmeier

Post by meik michalke
when the plugin starts with one of those two options and targenFile
is not set yet, you can't submit although i believe i've defined the
"required" rules accordingly. notably, you only have to scroll to any
other format option and back again to make the submit button
available. how can i fix that?

bug. Now fixed in git.

yes, awesome.

also the new overwrite checkbox in the browser control is exactly what i had
in mind.

did you have the chance to have a look at the most recent version of the
export plugin?


viele grüße :: m.eik

[Thomas Friedrichsmeier]

Hi,

On Wed, 24 Oct 2018 21:04:18 +0200

Post by meik michalke
did you have the chance to have a look at the most recent version of
the export plugin?

sorry, I delayed this, and now the seafile link has expired.

But why don't you just add it to our git repo. It's not like we cannot
apply any further tweaks once it's in. It's clearly something we want
to have in the main distribution.

Regards
Thomas

[Stefan Rödiger]

Hi,

I second the suggestion by Thomas.

I had the pleasure to use it. For me it was working will.

Wouldn't it be nice to have this available via the RKWard git tool you have developed?

Kind regards
Stefan

Post by Thomas Friedrichsmeier
Hi,
On Wed, 24 Oct 2018 21:04:18 +0200

Post by meik michalke
did you have the chance to have a look at the most recent version of
the export plugin?

sorry, I delayed this, and now the seafile link has expired.
But why don't you just add it to our git repo. It's not like we cannot
apply any further tweaks once it's in. It's clearly something we want
to have in the main distribution.
Regards
Thomas