Experimental Evaluation Charts
To evaluate the resulted PoC implementation, we devised and executed multiple scenarios where each was focused on one of the following four performance parameters:
- The manipulation of the number of participants in order to test the scalability of the service;
- The timing of the actions of the participants on the virtual objects in order to test the performance of the service in high or low concurrency;
- The fault-tolerance of the service, by introducing crash-failures in the system and observing how they affect the performance of the overall deployment;
- The network & processing capabilities effect on the service, when manipulating the location and processing capabilities of each of the participating servers.
Our findings can be observed through the various charts shown in the dashboard bellow. The dashboard is interactive, thus, one can change various scenario parameters at a time (i.e., Topology, Servers, Leaders, Type etc.) to further investigate the results. Please notice that the combination of some parameters may not have been executed as scenarios from our side yet, thus, yield a “No Data” message on the chart(s).
NVE Interface Executions Videos
Demonstration of an Execution of the NVE
What we see in the video is an execution of the NVE where the DSM service was deployed on three distinct nodes from Amazon Web Services (AWS). Using three nodes for the DSM, results in having in total three majority quorums with ids [q0,q1,q2]. In the scenario we have 5 Leader drones each one having 5 Follower drones resulting in total |L| + (|L|x|F|) = 5 + (5 x 5) = 30 participants in the service. The invocation scheme is stochastic, and read and write operations at each participant will be invoked at an interval generated randomly from the range [0.25…1]. The total execution time will be 120 seconds.
Follower Drone Coloring: Each time a follower drone completes a read operation, it changes its color to the color associated with the quorum it responded to it last.
Leader Drone Coloring: Initially a leaders color is black. Each time a leader drone invokes a write operation it colors itself with the blue color until the write operation terminates. Upon termination, it sets its color back to black until it invokes the next write operation.
Fault-Tolerance Demonstration
This video shows an execution of the NVE where the DSM service was deployed on five distinct Raspberry Pi nodes hosted by the Algolysis LTD (HO). Using five nodes for the DSM, results in having in total ten majority quorums with ids [q0,q1…q9] and assigned coloring as shown in Table 1. In this scenario we have 1 Leader drone being followed by 5 Follower drones resulting in total 6 participants in the service. The invocation scheme is stochastic, and read and write operations at each participant will be invoked at an interval generated randomly from the range [0.25…1]. The total execution time will be 120 seconds.
To visualize the fault tolerance guarantees of the underlying DSM service, in this particular execution, two replica crashes take place. In particular, right at time 0:45, replica serverA is killed. Replica serverA belongs to 6 quorums, with IDs [q0, q1, q2, q3, q4, q5]. Thus, right after the 0:45 minutes we can observe that no follower drone will color itself with the colors associated to quorums [q0, q1, q2, q3, q4, q5]. At time 1:45, serverB, that belongs to quorums [q0, q1, q2, q6, q7, q8 ] is killed. We observe that right after time 1:45, all quorums are faulty except quorum q9, thus, from that time on until the end of the execution, all followers complete their read operations by receiving replies only from the non-faulty quorum, q9.
| Q_ID & Color | Participants | Q_ID & Color | Participants |
|---|---|---|---|
| q0 | serverA, serverB, serverC | q5 | serverA, serverD, serverE |
| q1 | serverA, serverB, serverD | q6 | serverB, serverC, serverD |
| q2 | serverA, serverB, serverE | q7 | serverB, serverC, serverE |
| q3 | serverA, serverC, serverD | q8 | serverB, serverD, serverE |
| q4 | serverA, serverC, serverE | q9 | serverC, serverD, serverE |