1. Infrastructure Workflow

Cloudmesh supports an infrastructure workflow where users can specify python functions and map their execution on cloud infrastructure. The workflow feature allows you to define Python functions on a workflow class, and specify when to execute them via the command line or in a YAML file. You can then visualize the result of your workflow execution.

An example is given below.

from cloudmesh.flow.FlowDecorator import BaseWorkFlow


class MyFlow(BaseWorkFlow):
    def a(self):
        print("in a!")
        time.sleep(5)
    def b(self):
        print("in b!")
        time.sleep(10)
    def c(self):
        print("in c!")
        time.sleep(10)

This allows you to define functions in your workflow file. Then you can write a specification for the sequence to execute your functions:

(a; b || c)

Where

  • a ; b is executed sequentially
  • b || c is executed in parallel.

Finally, after execution the results are stored in MongoDB to be visualized or consumed by later functions in the series.

1.1. Javascript Interface (proposed)

We are looking for someone that would chose as its project to include a rendering of some DAG in javascript. The javascript library must be free to use. Nodes and edges must be able to be labeled.

A promissing start for a Javascript library is

This project is only recommended for someone that knows javascript already.

You will do the rest of the project in python. It is important that the functions be specified in python and not just Javascript. The focus is not on specifying the DAG with a GUI, but to visualizing it at runtime with status updates

Here is another summary that we posted earlier and is probably better as it has a dict return

So what we want to do is something i have done previously somewhere with graphviz, but instead of using graphviz we use java script. W want to define tasks that depend on each other. The tasks are defined as python functions. The dependencieas are specified via a simple graph string


 

def a (); print("a"); sleep(1) ; return {"status": "done", "color":"green", shape:"circle", label="a"}

def b (); print("b"); sleep(2); return{"status": "done", "color":"green", shape:"circle", label="b"}

def b (); print("c"); sleep(3); return{"status": "done", "color":"green", shape:"circle", label="c"}

 
w = workflow("a; b | c")

; = sequential

| = parallel


w.run()

While executing the javascript would change dynamically the state and color after a calculation is completed. The workflow should also be able to be specified in yaml

Here just one idea:

tasks:
    task:
      name: a
      parameter:
         x: "int"
         y:: "int"
      calculation: f(x,y) 
      entry:
         color: green
         label: a
         value: x (this is a python variable local to the function
         shape: circle
      return:
         color: green
         label: a
         value: x (this is a python variable local to the function
         shape: circle

Naturally at one point f(x,y) will be cloud related such as starting a vm and executing a command in teh vm ….

Followup:

We added a value to the return. Values can be any object.

def a():
    x = 10
    return {"status": "done", 
            "color": "green", 
            "shape": "circle", 
            "label": "c", 
            "value": x}

1.2. REST

An OpenAPI specification for this is to be defined.