Todo: Create Ingestion Policy

• We now setup Wendelin to receive data from fluentd
• Goto: http://softinstxxxx.host.vifib.net/erp5/portal_ingestion_policies/
• Security Setting to prevent arbitrary stream from being sent

Todo: Fast Input

• Hit the "Green Runner" for fast input
• Enter "pydata" as our reference name and save
• This creates a new ingestion policy and corresponding data stream

Todo: Check Data Streams

• Switch to Data Stream Module
• You will now have an empty stream, setup to ingest data from fluentd

Audio: Simulate IOT Sensor

Wendelin in Production

• First Wendelin prototype in production to monitor wind turbines.
• Wendelin used to collect data and manage wind parks.
• Machine Learning for failure prediction, structural health monitoring.

Wendelin in Production

• Turbine equipped with multiple sensors per blade, PC collecting sensor data
• PC includes fluentd for ingesting into Wendelin (network availability!)
• Tutorial: we use same setup, sensor - Fluentd - Wendelin

Basic FluentD

• FluentD is open source unified data collector
• Has a source and destination, generic, easy to extend
• Can handle data collection even under poor network conditions

Complex FluentD

• Setup of FluentD can be much more complex
• In Wendelin production, every turbine has its own FluentD
• Tutorial simulates the same, all VM have their own FluentD

Todo: Record Audio

• We use another SlapOS Webrunner hosting only a webpage:
• https://softinst56756.host.vifib.net/public/project/hyperconvergence/
• Click "Record" to Start/Stop audio and "Save" to save

Todo: Access Monitor using fluentD

• Access Monitor at: https://softinst67162.host.vifib.net/
• Go to the Editor, click "This Project"

Todo: Upload to Monitor

• We already made a folder for uploads for this tutorial
• Left click and select upload file

Todo: Forward File to fluentD

• In the Editor, open folders project > slapos
• Create a new file, give it your name
• Code on next slide (please copy&paste)

Todo: FluentD Configuration File (Gist)

    <source>
            @type bin
            format none
            path
            /srv/slapgrid/slappart9/srv/runner/PUT_YOUR_WAV_HERE/!!YOURNAME!!*.wav
            pos_file /srv/slapgrid/slappart9/srv/runner/!!YOURNAMEGOESHERE!!.pos
            enable_watch_timer false
            read_from_head true
            tag pydata
            </source>
            <match pydata>
            @type wendelin
            @id wendelin_out
            streamtool_uri
            http://!!URL_TO_YOUR_ZOPE!!/erp5/portal_ingestion_policies/pydata
            user zope
            password insecure
            buffer_type memory
            flush_interval 1s
            disable_retry_limit true
            </match>
          

Todo: Save and send from Terminal

• Switch to the terminal and enter:
• software/bd283b01bd665a31e41dce567f199491/bin/fluentd -c project/YOUR_NAME.cfg
• To pass your configuration including what file to send to Wendelin

Todo: Save and send from Terminal

• Head back to Wendelin/ERP5
• In the Data Stream Module, check the file size of the pydata stream
• It should now show a file size larger than 0