Under great applause the black cloth is lifted from the drone and you find yourself looking into two large eyes and sharply styled eyebrows. “The screen at the front of the drone responds to people's emotions. For now, it only responds to the emotion ‘happiness’, but we want to develop the software further for it to also respond to other main emotions such as ‘anger’ and ‘sadness’," explains interaction design engineer Isabel Leus. In addition, the drone can fly autonomously.

In the future, the team wants to develop even more opportunities for interaction with the drone, as well as better recognition of the environment and objects within it. Blue Jay began its journey in 2015, discovering what drones can do for us humans. From the distribution of beer in a café to the support of aid workers: it has been a serious journey.

This fifth Blue Jay team has made the drone smaller and thinner, among other things. The frame of the drone has become stronger through the use of a so-called T-beam construction and the shell is now easier to repair if a crash happens.

In the past, the team has already focused on the healthcare sector once and now we see similarities with the ideas for the drone to support aid workers. Will healthcare be the sector on which Blue Jay will focus after all? “No,” Smits says: “The third Blue Jay team has researched this, but it turned out not to be feasible because if you want to bring your product on the market there, it actually has to be 98 percent complete already. For now, we focus on the aid workers’ sector first, which is actually an interim vision to reach our ultimate goal: bring you your future drone assistant.”

A good help for the fire fighters

Whether the aid workers are hoping to get the help of such a drone? Oh yes. The fire brigade is present at the presentation and Mark Gijsels from the Veiligheidsregio Brabant-Zuidoost (the safety region Brabant-Southeast), which includes the fire brigade, is enthusiastic. “The drone could certainly help with prevention and thus the reduction of the amount of victims. I am specifically curious about what such a drone can do for healthcare institutions. Those often have to deal with understaffing, especially at night. In case of emergency, that is a problem. For example, if you have to evacuate a department in time - especially if you are dealing with people who are older or less mobile.”

A fire can quickly spread around, Gijsels outlines: “After three minutes, a normal fire can already be a raging fire. We have a response time and are there within ten to fifteen minutes. All time that you save or use to collect extra information is beneficial. In addition, it is of course helpful if the drone can turn out autonomously in case of emergency and can also communicate autonomously with us about the live situation.”

Gijsels also thinks further: “If a mini-extinguishing system can be installed on the drone that is able to extinguish a small fire, that would be great. It is certainly not a substitute for the fire brigade, but it could help us. For example when exploring situations: which areas are safe and which are not? How many people are present? What is their condition?" Blue Jay’s lead interaction design engineer Tijmen Poell: "In the future we want the drones to be able to communicate with the emergency services autonomously and also to be able to lead people outside via the safest exit(s), depending on the situation."

Applying it in the stadium

In addition to the general application of the drone for aid workers, applying it in the stadium is a very concrete example. A stadium is huge and many people can be present at the same time for big events. If there is an emergency, it can be difficult to quickly oversee the issue. If you want help from a drone, recognizing emotions in combination with behavior or gestures is important. An arm raised with the hand as a fist can be both a cheering supporter and a supporter who is ready to fight. If the drone recognizes the emotion shown on the face, a distinction can be made.

Interaction design engineer Fanni Egresits: "We want the software to recognize a hand gesture in a dynamic environment, not just on a white background, and we hope that the drone will soon be able to fly according to hand or body gestures." The team has researched this. In MetaForum, passers-by were asked how they would try to attract the drone’s attention if they wanted it to come closer. That output was then tested in a broader target group through a survey, which yielded a gesture that a large majority perceived as logical for beckoning a drone. The system will get this information and can respond to it in the future.

Software engineer Rahul Ramesh also talks about the possibility of weapon recognition by the drone: “With artificial intelligence this becomes possible. We used a dataset with different images of guns and trained the system to recognize such weapons. The proof of concept is not yet entirely successful; the weapon recognition works, but is not very accurate yet. We will continue to work on that.”

The future of the drone

Smits: “The biggest challenge is the size of the drone; it is still too large to fly indoors properly. The air that gets displaced, returns to the drone and causes it to crash." Finance & PR manager Marieke Vermeulen: "In addition, the drone still produces too much noise to be an assistant in a home environment."