- Education , Student
- 15/12/2020
What makes you ID worthy?
Need for clearer assessment criteria Industrial Design
Industrial Design students want clearer assessment criteria instead of ‘guidelines’. What makes you ID worthy? It's one of the questions Ieva Vaitiekunaite struggled with, as after years of assessments with good results she unexpectedly failed her final project. And she appears not to be the only one. The perhaps most creative TU/e program raises questions among students about assessment criteria and job opportunities. Program director Miguel Bruns tries to answer them.
In the summer of 2020, Ieva Vaitiekunaite's bachelor came to an abrupt end. “Industrial Design, the department that is joked about, because we are the 'fake engineers'. And now that I have been around for longer, I understand why," the former student says. "I've seen how it works: you aren't necessarily judged on your skills. There is always this large subjective part of your grade that seems to depend on your teacher's beliefs. Evaluation criteria are metrics that can’t be measured. For example: one of the criteria evaluates how much guidance the student needs during the project. There is no number saying how many times asking for help corresponds with what grade so 'much' to one teacher is not the same as 'much' to another one.”
Patrick Verhagen (not his real name) also came across this issue. Because of his job, he wants to remain anonymous. “I was looking for an internship for Industrial Design and it was difficult to find something suitable. With a bit of fuss I could start working for a company for financial consultancy and risk analysis. An internship must be signed for and arranged before the start of the third year, so no later than the summer of the second year. I had arranged that on time and it turned out that I had not passed the project that preceded the internship. That surprised me very much. I had gone through an entire research process together with my group, aimed at eventually publishing the result.”
“Our project supervisor was very enthusiastic and wanted me to pass, but that turned out to be impossible due to my reflection. It may not have been fantastic and I asked if I could fix it, but that was not possible. I have accepted that, but I personally think that a reflection should be less important in the assessment than the product you deliver. I used the summer to overthink things a bit. Then I got an email from the Association for Computing Machinery Inc., a publisher. They wanted to publish my group’s paper! That does indicate that our paper was of a sufficient level. That made it feel extra strange to have failed because of my reflection. I decided to do my internship anyway, even though it could not be used for my bachelor Industrial Design, due to failing the project. The company was so satisfied with me as an intern, that I was offered a job. I accepted it and now I work on communication and automation.”
Miguel Bruns, associate professor and program director of the Department of Industrial Design, responds: “A project can be excellent in one area, for example computer science. It could even be published as a paper in a journal, but still not pass for Industrial Design because the different areas of expertise are insufficiently integrated” - on which more later in this article.
The Rubrics: a guideline without obligation
“Of course it is a subjective experience, the assessment”, alumna Lidewij Heerkens starts. “I don't know if this applies to everyone, but I thought my assessment process was a bit strange. I failed my final project BEP, but then I did pass the resit, luckily. I did my BEP at innovation Space (iS) in a multidisciplinary team, but I was assessed at ID. There was a lot of confusion in the communication at ID and then corona came to top it off. I was working on my project and a week before the assessment I was told by my coach that it was not good enough: I got a 5.0. That was interesting, because in the Rubrics booklet everything was marked ‘sufficient’, except twice ‘insufficient’ and three times ‘good’ (pages 39 to 42, ed.). ” That Rubrics booklet(this is the version that applied to Heerkens, the version from October 2020 is now online, ed.) describes the required developments and the expected competences for an ID graduate.
“A simple calculation of the above marks shows that this combination must still yield 'sufficient' on average. I made inquiries and my coach then said something about the grade being set for the overall project and in comparison to other students. After the resit I suddenly got a 7 and that was three weeks later. I thought that was quite a big difference for three weeks after a six-month project. How they came to the 5 or 7 is a mystery to me. The Rubrics is a kind of guideline and not binding, your coach can deviate from it. The pages from 43 onwards have never been completed, no idea why they exist and what they are used for. How far the guideline reaches, whether that differs per coach, how the grade is created: I have no idea. How many times does which part of the Rubrics count for the grade? When is a prototype or research ‘good’? Clear assessment criteria, instead of guidelines, would be nice."
No weighted average
“I conducted research on ethics from a design perspective and created a framework. The assessor found that my framework did not meet the requirements for the demonstrator. Ironically, I got a bottle of lemonade sent to my house from iS as a prize for the project I had done because it was the best project of the year, while I was still doing a resit. I have repaired the case and that is also not the problem, it is about the lack of clarity about the assessment and the information,” Heerkens says.
Bruns: “Projects are given a verdict, a Pass or Fail, and in exceptional cases (the best 10%) an Excellent. Bachelor's final projects are assessed with a grade, but this is not a calculated average. It is not the case that if you have a ‘sufficient’, an ‘insufficient’ and a ‘good’ in the Rubrics, you automatically get an average ‘sufficient’. The Rubrics are not matched one-on-one to a number of errors, as is possible with calculus. They are supportive, not guiding. It is always in relation to the performance of other students. If several students have the same assessment results in terms of times they have received ‘good’, we look at how the projects differ and we decide which grades go with that."
Not the right skills
"I did my graduation project at Signify," Vaitiekunaite says. The university gave the green light "and even recommended me to do my graduation project in a company: 'Signify has more time to guide and support you than we do with the number of students in your year,' I was told. At the end of the project she had to present and suddenly the attitude of the university assessors was very different: Vaitiekunaite failed her graduation project. She was amazed: "According to the teachers, the main reason I failed without a chance for a reassessment was because I didn't have the balance of skills that a bachelor's of Industrial Design needs. That's shocking to hear after three years of bachelor's studies. This part was never communicated to me before the final moment of assessment."
To appeal
To make this even weirder, the company was very satisfied with Vaitiekunaite and even offered her a job, which she took. “Of course I would like a diploma, but I cannot pass up such an opportunity, and after all, I want a good job after my diploma. But I don't want to just let this go, either. Then I decided to file a complaint.” That turned out not to be so easy: if she wanted to not just object but go for appeal, she was told to register again (for cohort 2020-2021, ed.). “That's ridiculous, isn't it? It happened in the summer of the 2019-2020 cohort and I just need to be able to file a complaint and be kept informed, instead of paying hundreds of euros to get clarification at all. In the end, I wasn’t stopped after starting a procedure without registering again.” The exam committee responded with, "The teacher has followed to the protocol." But that was never my problem and my arguments were not about that. To the day of today, my situation is still being ‘analyzed’.”
Heerkens: “I consciously chose not to appeal against my 'conditional', but to take the resit. I didn't feel like putting time and energy into that fight. Imagine you think something is judged too subjectively; I would not know on what ground I should request a second opinion. If you have concrete criteria, you can appeal.”
Not an exact science
“Industrial Design (ID) is not an exact science, like mathematics,” Bruns says. “ID is characterized by the integration of different disciplines. We train students to look at the design process from five different areas of expertise. The importance of these five perspectives is stressed to the students from an early stage. In the bachelor we offer the students different courses to develop those perspectives. In subjects such as calculus or statistics you can clearly say in the assessment whether something is right or wrong, but in the assessment of philosophy, art or the process itself, there is a bit of subjectivity. We also need to assess how well students have integrated their vision of design into the design process. At least three of the perspectives must be represented. Some students are insufficiently aware that they need to develop these five perspectives, but (choose to) look from the perspective of their preference. For example, if you do that mainly from a computer science perspective, it is better to study computer science. The added value lies in the integration of the ‘lenses’. "
Having to explain your expertise to employers
If you still have to explain your studies to potential employers or companies to do an internship, it can make finding a job or internship more difficult. Vaitiekunaite noticed this when looking for an internship. “An average HR employee does not know what Industrial Design is and cannot imagine what an ID student can do and whether that fits the company. This is especially the case with large companies where HR makes the first selection.”
Verhagen: “At first my internship company didn't really understand what Industrial Design concerns, I really had to explain that. We (industrial designers, ed.) can do all kinds of things, but compared to other studies we do not have any real specializations that offer realistic opportunities on the labor market, I think. There will probably be students who see it differently. I got my internship through my own initiative and network, it had nothing to do with the university. The university gave a few suggestions, but had I used them I would have never been able to find anything like I did. Industrial Design is really a good program in terms of content: I learned things that I couldn't have learned anywhere else - for example moving perspective. I’ve noticed in my work how valuable that is: you have to be able to take a different perspective in order to create good solutions. You also learn to deal well with people. And without this incident I would have finished the program. But it took the fun out of it.”
At first my internship company didn't really understand what Industrial Design concerns, I really had to explain that
Independent assessment of the ID program
Independent assessment of the ID program
The ‘Keuzegids Universiteiten 2021’ (Catalogue Universities 2021) indicates that TU/e’s Industrial Design program scores average as a final assessment and scores three points better (61) than the more 'structured' variants at TU Delft (58) and University of Twente (58), although the reviews are not far apart. The testing in Eindhoven is rated below average. Furthermore, there is a relatively large freedom of choice in Eindhoven because of the electives. Although the field of ID as a whole is assessed as reasonable, the job opportunities up to 2024 are assessed as poor. The catalogue states “Finding a job is fairly easy, but is nevertheless more difficult than with other technical studies. Barely half of the graduates find quality work. As a successful designer you can earn well, but on average the salary is lower than for the 'harder' technical studies.”
Delft versus Eindhoven
When it comes to job searching and finding internships, Bruns recognizes that there is room for improvement, but he also tries to make students aware early on about how to use their portfolio to increase their opportunities in the labor market. “At large companies such as BMW or Canon, they know what we do. We see the challenges more at smaller design agencies, especially if they are led by designers with an ID background from TU Delft. That is really different from Eindhoven, which leads to different expectations. Delft and Twente are more alike. We are different in Eindhoven because we made the link with electronics and computer science quite early on. Interaction between humans and computers, et cetera. Delft traditionally focused more on industrial product design and mechanical engineering, although that has changed somewhat over the years. We try to explore the edges of the field, so you are ahead of traditional design agencies.”
“Delft has a much more defined profile, but we are now trying to establish a clearer link with the wishes of the business community in the courses we offer. For example, UX designers are in high demand. We have been offering courses on this topic for a long time. When thinking of ID, employers often think of product design such as ID was introduced in Delft fifty years ago. We cannot control that, but we are now making the profile clearer. We also work together with the business community, for example with Philips and Canon. But students often think that they are finished after the bachelor, but the real in-depth work happens during the master. We find it important to communicate that clearly.”
Match the profile
Heerkens: “I obtained my bachelor's degree with an average score of 7.3. If you see my bachelor background, you would think I should be able to pass my final BEP. Now I am doing another master's at another university. And yes, how things went during my bachelor at ID, was a factor in that decision. I like a little more clarity about grades and assessments. There is too much uncertainty for me about how you will be assessed and when a project is good enough. In my current (pre) master's, that is clear."
Now I am doing another master's at another university. And yes, how things went during my bachelor at ID, was a factor in that decision
'How is it possible to have ‘a lack of the right skills’ after three years of education?,' Vaitiekunaite wonders. “Thirty percent of our education concerns free electives. Then the department says; choose what you like, as long as it suits your own professional development. But when you do, you get punished for it. I completed my entire curriculum with good and excellent assessment(s) and yet I failed. Then there is something wrong with the curriculum.”
I completed my entire curriculum with good and excellent assessment(s) and yet I failed. Then there is something wrong with the curriculum
Career opportunities
Verhagen: “After your studies you get a diploma with Industrial Design on it. If an employer sees that, you still have to prove yourself. How different is that if you have studied Electrical Engineering. Then you have concrete skills, which require little further explanation. Your career opportunities are really different as an ID graduate. In my opinion, this is not so much because of the program itself but because of the structure around it, such as the communication and assessment method. There are gaps and that, for me, is the point I want to make: if you pay more than two thousand euros in tuition fees per year, I think you may expect a little more structure and communication,” Verhagen says. “The structure has to be redefined: what is Industrial Design and what can you do with it? If you choose ID, you really have to specialize and develop your own style to have a good chance on the labor market. I think it is important that this becomes clearer, starting from the orientation phase of potential students,” Verhagen says.
“I hope that I can still get my diploma,” Vaitiekunaite says. “Even though I can do my job and I get a good salary, I still want a diploma after three years of hard work.” Vaitiekunaite has been working as a social media manager for a number of months now and is enjoying herself. "I am very happy that I have decided to accept the job."
Increasing your chances to find a job as an ID student
"We ask students to create a clear profile and portfolio", Bruns says. "They have to communicate their vision of the field well. The problem is that students see the courses in the study program as things they need to tick off the to-do list, but they need to integrate the knowledge and skills they acquire in those courses, into their vision. And there is an area for improvement in coaching and supervision of students. I give an annual lecture for study association Lucid on how students can present their profile better. I say: “Check what companies need: don't just respond to ID positions." I give them examples of different job profiles they can apply for, such as UX designer, IoT designer (Internet of Things designer, ed.), Data-enabled designer, et cetera. And then indicate which courses you have taken that demonstrate your expertise. The students must clearly show what they are capable of.”
Discussion