Diaries for Statistical models for network communication
This is a place for collecting information about the project.
Our main partner for this project is Professor George Porter.
- The paper on Pivot Tracing can be found here: http://dl.acm.org/citation.cfm?id=2815415
- the project web page is here: http://pivottracing.io/
- Code can be found here: https://github.com/brownsys/tracing-framework
- There is no support for Spark yet, but there is support for Hadoop Map-reduce. We might want to use that. Need to find out if we can use Pivot Monitoring on AWS.
- Slides for the paper and our proposed scope: https://goo.gl/NtNokV
- Email from Jonathan Mace regarding AWS install
- Email from Jonathan Mace regarding compiling queries
The MSc. Students taking part in this project are:
- User:Praveen Origanti
- User:Tsunghan Lee
- Emails: Shen Ting Ang <firstname.lastname@example.org>, Praveen O <email@example.com>, Jayant Malani <firstname.lastname@example.org>, Tsunghan Lee <email@example.com>
- Finished StarCluster tutorial.
- Tried to deploying StarCluster, but some issues appears, waiting to be solved.
- Updated StarCluster's Hadoop plugin to work with pt_hadoop AMI (ami-02f6a115 on us-east-1)
- Able to create a Hadoop cluster using this plugin. However, MapReduce tasks do not work (as yet) due to ConnectionRefused errors
- Instructions to create a PT-Hadoop cluster using the updated StarCluster. Need some help fixing this.
- Created StarCluster AMI (ami-6ebbec79) with Pivot Tracing/Hadoop installed. Works off Ubuntu 16.04. Tested with the example PT query.
- Updating StarCluster's Hadoop Plugin to be able to create a cluster on this AMI
- Team Notes: Sampath unavailable/busy for the next 4 weeks. Shen Ting is not certain of committing ~10 hrs per week.
- Summary of work up to June 2016
- Get an account on this wiki and keep it updated
- Decide how much time you want to devote to the project starting in the fall, do you want to do it as a project or a thesis.
- We have successfully installed Pivot-tracing instrumented Hadoop, following the PT tutorial on AWS
- We are working on a script, that can be run on a new EC2 instance, by any team member