Terje Brekke

Experience-based Master in Information Security

Erjon Zoto

There were different ongoing challenges and issues pending from 2021, which could be mentioned here. The pandemic was over during Spring 22, but the challenge of having MISEB students on campus persisted, as they were still working full-time. Having more sessions on campus or adapting the online sessions to MISEB students has been perhaps the major challenge, which is still not solved at the optimal level. However, on-campus meetings are still part of the mandatory activities for all MISEB students, and this will not change for the year to come.I would then start by mentioning Handlingsplan from last year, along with the activities done so far: <ol><li>Plan for more events on campus</li><li class="ql-indent-1">MISEB klassemøte, April 22</li><li class="ql-indent-1">Welcome day seminar, August 22</li><li class="ql-indent-1">Klassemøte torsdag 3. nov (planned)</li><li class="ql-indent-1">Work in progress</li><li>Plan for better teaching quality</li><li class="ql-indent-1">Work in progress</li><li class="ql-indent-1">Specific courses have specific settings</li><li>Plan for meeting with course responsibles</li><li class="ql-indent-1">Teacher's meeting, Spring 22</li><li class="ql-indent-1">Teacher's meeting, Fall 22 - 1</li><li class="ql-indent-1">Teacher's meeting, Fall 22 - 2 (planned in November)</li><li class="ql-indent-1">Work in progress</li><li>Plan for meeting with KK</li><li class="ql-indent-1">First meeting, January 22</li><li class="ql-indent-1">Many informal meetings as IIK colleagues</li><li class="ql-indent-1">Next formal meeting - December 22</li><li>Plan for meeting with KTV</li><li class="ql-indent-1">Meeting with student contact, April 22</li><li class="ql-indent-1">Meeting with student contact, August 22</li><li class="ql-indent-1">Next meeting in November/December'</li><li class="ql-indent-1">Work in progress</li></ol>All planned actions were started in due time (Spring 22, as planned), but there is still room for improvement, as we are in the middle of the periodical evaluation process. It is still challenging to meet and discuss with MISEB students and their representatives in person, due to the group specifics and the busy schedule on both sides. The actions have had a good effect in raising awareness about ongoing issues and challenges with the study program overall.Something to be added here is that there are specific courses within MISEB where on-campus presence is asked in exchange for a suitable offer in terms of a workshop, good facilities for individual and team-based learning, guest lectures etc. The challenge is then trying to replicate this with the other courses so that students might find it even more interesting to show up on campus and spend perhaps 3 days or a full week with different activities, rather than a 1-2 day period, once a semester. Other inputs included in the 2021 report:Input 1:<ul><li>More practical, hands-on experience, in combination with lectures and other teaching activities</li><li>The labs with practical experience should make use of mandatory presence on campus</li><li>Invest and train for better quality tools and teaching activities</li></ul>There is work in progress for all the points mentioned above, some specific courses have included more online sessions to make up for the students attending remotely. More work is needed in this direction. Pedagogical development courses could be a solution, especially for the newly appointed teaching staff or for those supporting the main teachers. Input 2:<ul><li>Follow up with the transition to more courses available as EVU</li></ul>This is an ongoing challenge, based on the fact that the production numbers for MISEB are not very high (compared to the average) This could change if we ensure more courses are available as EVU, so that fewer students opt for a Master's degree for attending the same courses. The topic is still under assessment and will be part of the ongoing meetings with the course teachers and KK

Opptak og studiestart: Recent numbers show that the MISEB program has a high relevance among the main target group in Norway, as shown in the attached report from BEVISST. The number of applicants and those that have MISEB as 1st choice have been increasing on a yearly basis, especially for the 2021-2022 year. This is surely one of the strong aspects of this program, ensuring its endurance for at least a few more years. This also means there are no big challenges for the recruitment process at this stage, which should support the further development of the program.As for the study start for the MISEB students, the welcome seminar was organized as a Zoom meeting in August 2021, due to the ongoing pandemic situation and a few international participants. The event however went as planned, and many students used the final part of the meeting for asking questions to the administrative representatives. During the session, the students were introduced to the SPL in charge and its supporting team of cohort coordinators. The students were also able to understand more about the main courses and about the master thesis process.A challenge to be mentioned here could be the lack of involvement of the SPL or other program representatives within the admission process, something which has not yet been duly addressed. Læringsmiljø (fysisk, psykososialt, organisatorisk, digitalt, pedagogisk): The numbers from Studiebarometer show the students appreciate the learning environment created around them., especially from a pedagogical perspective. On the other side, the ongoing pandemic in 2021 affected the social environment around them, which had the lowest score, understandably. The pandemic allowed for increased digital interaction, and the MISEB students were positive about the usage of online facilities for the different course activities.Once the pandemic was over, the MISEB teaching staff succeeded in improving the social aspect, with more meetings and activities on campus. This will hopefully be reflected in the next Studiebarometer, as the students should benefit more easily from the ongoing activities on campus. Sammenheng og progresjon: As already mentioned, the production numbers for MISEB are not good enough. However, this can be associated with the specific context, as more and more students tend to get their degree after 6 semesters or more. There are also success cases, where students are able to fulfill their studies in less than 2 years after starting, as a few of the courses are recognized from before.During 2021, according to Studiebarometer, the MISEB respondents have had less workload and learning activities offered by the program. This is mostly related to the pandemic situation, while the numbers could change in the following period.There have not been huge challenges related to specific courses, but there is room for improvement, i.e. when it comes to the preparatory courses before the Master's thesis. Relevans: The program's relevance is one of the strong aspects of MISEB. The program is getting a high number of applications on a yearly basis, and this supported our decision for increasing the number of places for the next academic year. The relevance is incorporated in the specific courses that are tailored to MISEB students, while there is a good collaboration with the industry, expressed in terms of guest lectures, projects, and theses that are handled in close cooperation with them. Internasjonal orientering As for outgoing students, there is the possibility for all MISEB students to apply for continuing their studies outside Norway, but in practice, there were almost no applications last year, as students are also full-time employees, so they are not focused on using this opportunity.On the other side, the teaching staff for MISEB is composed of international members, some of them only engaged with a specific course, which helps the students get a broader perspective on the different topics of interest. Most of the teaching activities are based on research-based teaching, which also helps provide the latest developments outside NTNU and brings them closer to reality. Research-based teaching and up-to-date facilities and tools are also relevant factors for attracting the next generations of students, so this is also the main focus of our agenda.

The MISEB study program has been offered by a stable and experienced group of teachers throughout the past years of its existence. The students, with some exceptions during the pandemic, have continuously had a high appreciation of the learning environment and requirements of the teaching staff in the different courses. This context has helped us support the idea for increasing the number of places for incoming students in the next academic year.However, there is room for improvement, both in the quantitative, and qualitative aspects. The teaching staff is composed of multiple members, ranging from førstelektor to professor, with a balanced combination of temporary and permanent staff, including Ph.D. students and postdocs as supporting staff. The program will need clearly more resources for ensuring a sustainable process in the coming years, as members of the teaching staff are going to retire or might relocate and hold temporary positions within the department. Thus, the main challenge in finding the right people to replace the outgoing staff or in increasing the number of those existing should persist. There could be also a need for more courses to be offered, which would then ask for more resources to accommodate the MISEB students' needsThe other aspect is related to teaching quality in general. This could be addressed by allowing the teaching staff to enroll in training and workshops related to pedagogical quality, and perhaps renew or improve their skills, as related to up-to-date tools and research done in the field. This process should be discussed and agreed upon with the institute and the faculty.

Consider having more activities on campus that would be highly useful for the students, in terms of improving the social and pedagogical aspects around the program they are attending. This could mean having more courses offering physical (mandatory) sessions on campus, combining activities together and providing students a variation of options during their stay in Gjøvik.

Tiltak

Ensure more on-campus activities

Make sure to find a structure that helps most of the students to easily progress through the different years in MISEB. This could relate to courses being taken at a certain level, master thesis preparatory courses being mandatory etc.

Improve individual progression

Improve social aspects

Assign KTV for MISEB

NTNU:TIA:REST

Faculty of Information Technology and Electrical Engineering

Fakultet for informasjonsteknologi og elektroteknikk

The follow-up plan from the previous academic year had 2 main measures, which will be addressed below:<ul><li>Ensure more on-campus activities</li></ul>		In this case, based on a closer collaboration with the MIS program responsible and the core team, we were able to schedule and organize more meetings with our students and teaching staff. As an example, we did have a meeting in December 2022 with a festive theme around Christmas and New Year on campus with students present from all MIS programs. More meetings were held on campus or digitally, as needed. This measure is considered as fulfilled, however it will be evaluated and addressed on an ongoing basis. We still need to establish a better structure for the yearly activities with our students, overlapping with the activities that the Faculty and NTNU promotes. A challenge here is related to the budget available for such activities, but this is expected to be solved soon in coordination with the Department and the Faculty.<ul><li>Improve individual progression</li></ul>		To fulfill this measure, we started by assigning the pre-Master thesis course (Research Project Planning, IMT4205) as mandatory for all MISEB students starting in Fall 2023. This was intentionally done to help the students plan better their final semesters and the work for delivering the master thesis. There are no available results yet, but we expect this measure to be very useful for the students and the numbers for the MISEB program in general. The measure is considered as fulfilled, however, as for the other measure, we will follow up with more steps if needed.		Another aspect here is related to the overall structure for the MISEB program. There is already work in progress around the new study plan/structure for the MISEB program, while a task group was recently formed to deal with this process.

Opptak og studiestart:The MISEB study program is still following recent trends, with a high number of applications, as compared to the study places available. For 2022/23, there were 140 applicants, with more than 80% of them having the MISEB program as 1st priority. This is now becoming the trend, which also pushed us to ask for an increased number of study places for the incoming academic year (from 20 to 30). Another important development in the last year was the increased number of applicants who confirmed for the final spot (møttall), which arised to almost 70, or 3.3 applicants per study place. This was certainly an exception from the usual trends, and it reflects the opportunity that many students from the other MIS programs used to continue their studies at NTNU Gjøvik. We do not expect this case to be repeated in similar numbers in the next year.The good numbers that MISEB has had in recent years about incoming students have not pushed us to think or plan about specific recruitment strategies for the program. However, there might be room for improvement, especially when it comes to getting more female applicants, so that the gender balance is better preserved at the final stages of the process.As for the study start/welcome day, the MISEB students, due to their specific requirements, are offered only a day of mandatory activities, where the MISEB core team and the teaching staff is introduced. For the 22/23 academic year, the seminar was held on campus for the first time since the pandemic. The activities were planned jointly with the other MIS students, but there was also room for separate sessions, as MISEB students can attend some courses tailored specifically for them. The whole seminar was introduced and announced well in advance, so that all incoming students had time to plan their trip and presence in the meeting. For the next round, there were changes planned, aiming towards higher interaction between the student groups. Læringsmiljø (fysisk, psykososialt, organisatorisk, digitalt, pedagogisk):For this part, numbers were considered from both the results of the Studiebarometer survey, as well as the different meetings we had with the students. The learning environment was still a challenging aspect for the MISEB students last year, especially the social aspects around it, at least for those participating in the different surveys. The numbers went down from the previous year, but still in line with the overall number from the different programs in Information Security, as Studiebarometer reports show.As mentioned already, one of the measures here was to organize more meetings, especially informal ones, with all MIS students together, in order to get them to know better each other and the different facilities offered at NTNU Gjøvik, especially for the students off-campus. We planned for some changes in the welcome day as well, so that the students get more interaction from the 1st day on campus.Another thing to mention here is the effect that the presence of student representatives would have on such activities. As we speak, there is no student representatives for the MISEB program, but we were able to have a PTV for all MIS programs, along with the elected ITV.We have no results available yet, but we expect better results than the ones provided from the 22/23 year, especially as they were answered from students starting their Master studies in the middle of the pandemic. Hence, this measure will be addressed on an ongoing basis, at least until we get high consecutive numbers from our students. Sammenheng og progresjon:In the past academic year there were positive developments as related to students fulfilling their master study in the MISEB program. We had 13 students graduating, same number as in the previous year, also the highest amount in recent years. This means production numbers for the MISEB program are improving, though we are still below 20% of all students starting the program overall. As for students failing courses or withdrawing from their studies, the numbers are stable or slightly improving, as compared to the previous year. This is of course subject to change on a yearly basis, but the results are promising.As for the general progression, the average MISEB student still requires more than 6 semesters to fulfill his studies, however the average is going down recently, with some special cases finishing their studies within 4 semesters or earlier. Considering that the MISEB students are working 100% besides studying, these numbers are not that alarming at this stage.When asked about their own progression, the MISEB students were happy with their progression, admitting they put more effort than their peers from previous studies. This is also supported from the numbers from the Studiebarometer survey showing that the respondents have committed more time to the different activities for the MISEB program. These numbers are similar to the numbers before the pandemic (2019-)Finally, the feedback received from several meetings with students and teaching staff has not provided specific challenges with the way how courses are integrated together in the MISEB program. However, the ongoing challenges when considering all MIS programs together pushed us to start planning for new study plans in the near future. Relevans:The numbers mentioned already provide strong evidence of the relevance that the MISEB program has for the working community, however there are different signals arriving from the students. In the Studiebarometer survey for 2022, when asked about the program's connection to the working environment, the respondents for MISEB provided slightly lower numbers on average than the year before (from 3.2 to 3.0), reaching average numbers. Though the number of answers does not represent all MISEB students from kull 22, the feedback needs to be taken into account when planning for the coming years.When it comes to how the teaching staff and students are put in contact with the industry, there are multiple cases of such activities, such as guest lectures, projects, and up to final thesis supervised from internal contacts at the specific companies. External guests were present in the last welcome day seminar, and there will be more such cases in the upcoming activities. Internasjonal orientering The MISEB program is offered to working students, hence the option for travelling outside on a minimal period of 6 months or more might not be on their highest priorities. However, all MISEB students are eligible for the same exchange opportunities that all NTNU students have. Actually, there was 1 student applying for the exchange program last academic year. He is now following a 1-year program in Japan, based on the corresponding credits. Information on exchange opportunities is available on the NTNU pages, but not in the main focus of our program's information page.On the other side, the teaching staff for the MISEB program is largely international and spread in the different semesters. This allows for a large international experience, as well as opportunities to benefit from a large pool of international contacts in the desired fields of study and research. In addition, research-based teaching is the essence of our activities in the MISEB program, which extends the interaction with state-of-the-art techniques and methods even further.

As mentioned in previous round of the evaluation report, the MISEB program benefits from a wide range of experienced teaching staff, with relevance connections with the industry and the international community. The students' feedback was positive when it comes to the quality aspects of their interaction and feedback received from the teaching staff.The new structure around the study programs has enabled closer collaboration with the administrative staff and teaching staff that are relevant to our MISEB students, and this has helped us understand better the needs from both sides and perspectives. As for the teaching staff, while expecting more students to attend the different courses, there is a need for addressing better the resources available and helping them develop their pedagogical skills. There are several courses being taught by single teachers, which might need further support, as well as other courses being taught by newly employed staff. In addition, we need to account for staff members relocating or retiring, as well as for new potential courses to be taught. All these aspects need close collaboration and coordination with the staff members, the department and the Faculty representatives.

On a general level, the results from Studiebarometer 2022 show lower level of Satisfaction (Tilfredshet) than the previous year - from 4.3 to 3.7. This brings us back to numbers from before the pandemic, which is not the intended way forward, but still in line with the overall results from other programs in Information Security. The overall results show also negative developments when it comes to the Learning environment (especially social aspects), Connection to working life, and less on Organization and Assessment forms. On the other side, the highest score was given to Teaching, while Feedback had also noticeable positive change from last year (highest in 4 years), with Own engagement having the largest increase from last year.Another indicator included in the last Studiebarometer survey involved the use of digital tools. The results from the MISEB respondents show that, while the teaching platforms (Blackboard, Canvas etc.) and approaches (streaming, videorecordings) are highly used and relevant for them, they also see as relevant the use of collaboration tools (Drive, Discord), interactive tools (Mentimeter, Kahoot), followed by simulation tools and course-specific tools (Excel, Python etc)The results shown from the Studiebarometer survey, along with the feedback from the meetings with teaching staff, students and student representatives, will help us find the relevant aspects to address in the coming year/s. In addition, it is expected that a new MISEB plan will be available and running from Fall 2025, with relevant changes from the current or ongoing study plans. Note: Files attached do not include Studiebarometer results, due to issues with Tableau. Relevant indicators and numbers were described in the report and conclusions above

Experience-based Master in Information Security - MISEB