FAIR, fast, and frictionless – computational workflows with SciWIn

FAIR, as , and ic ionless –compu a ional
wo k lows wi h SciWIn
Jens K umsieck1, An onia Leidel2, Xa e S iensmeie 3, Ha ald on Waldow1
1: Johann Hein ich on Thünen-Ins i u e, B aunschweig, Ge many
2: Leibniz Ins i u e o Plan Gene ics and C op Plan Resea ch, Ga e sleben, Ge many
3: Biele eld Uni e si y, Biele eld, Ge many
Scien i ic Wo k low In as uc u e
CSV
md
{ }
json
>s4n c ea e
–c py hon:3.13 --enable-ne wo k
py hon analysis.py
--da a aw.cs
--s ic ue
Me ada a
Inpu Pa ame e s
Ou pu Pa ame e s
Run ime En i onmen
Command
acks ile changes
gi Reposi o y
CWL
Added in commi
CWL CommandLineTool
>s4n connec my_wo k low
-- om aw_da a
-- o analysis/da a
Me ada a
Inpu Pa ame e s
Ou pu Pa ame e s
Wo k low S eps
>s4n connec my_wo k low
-- om analysis/da a
-- o esul _da a
CWL
CWL Wo k low
>s4n isualize
wo k lows/my_wo k low/my_wo k low.cwl
cwlVe sion: 1.2
class:CommandLineTool
equi emen s:
-class:Docke Requi emen
docke Pull: osgeo/gdal:ubun u-
ull-3.6.3
inpu s:
-id: ilename
ype:s ing
de aul : dis ic s.geojson
inpu Binding:
posi ion: 0
ou pu s:
-id: dis ic s
ype:File
ou pu Binding:
glob: $(inpu s. ilename)
baseCommand: og 2og
cwlVe sion: 1.2
class:Wo k low
inpu s:
-id: oken
ype:s ing
ou pu s:
-id: aw_da a
ype:File
ou pu Sou ce: collec / aw_da a
s eps:
-id: collec
in:
-id: oken
sou ce: oken
un: ../collec /collec .cwl
ou :
- aw_da a
CommandLineTool Wo k low
Me ada a
Inpu
Pa ame e s
Ou pu
Pa ame e s
Run ime
En i onmen
Command
Me ada a
Inpu
Pa ame e s
Ou pu
Pa ame e s
Wo k low
S eps
CWL Building Blocks
Common Wo k low Language (CWL)
Au oma ed
c ea ion o CWL
CommandLineTools
C ea ion o complex
CWL Wo k lows by
connec ing Tools and
Wo k lows
THE PROBLEM
•Rep oducible compu a ional esea ch is challenging
•Scien i ic compu a ional wo k lows as a solu ion
BUT:
•Mo e han 300 Wo k low Managemen Sys ems o
choose 🤯
•Resea che s spend hou s o w i e boile pla e code
o Wo k low de ini ions 😵
•Open s anda d o wo k low de ini ions
•Me ada a desc ibing wha no how o un
•Based on YAML (machine & human eadable)
•Po able
>s4n execu e emo e s a
wo k lows/my_wo k low/my_wo k low.cwl
inpu s.yml
KEY FEATURES
•Command-line applica ion (s4n), w i en in Rus
•Au oma ed gene a ion o CWL Files
•Wo k low Visualisa ion (publica ion eady igu es)
•Ve sion-con olled wo k lows using gi
•Easy euse exis ing Wo k lows ( ia gi submodules)
•Local execu ion o CWL Files
•Remo e execu ion using Reana,
in coope a ion wi h de.NBI
•Expo eana uns as p o enance un c a e
•Compa ible wi h Da aPLANT‘s ARC o ma
COMING SOON
•Wo k lowHub.eu in eg a ion
•(Semi-)au oma ed Docke ile c ea ion
•G aphical Use In e ace
>s4n execu e local
wo k lows/my_wo k low/my_wo k low.cwl
inpu s.yml
eana.bi.denbi.de
REST API
Wo k low Execu ion
local o a scale
gi hub.com/ ai ag o/m4.4_sciwin_clien
Iillus a ion by h ps://www.manypixels.co