0x010088F6312065A70A44B630A6848498EAD80093CFEAFE161DC94D83C7C5C9A5A748B8

My main goal this year was to switch from a normal "sage on the stage" lecture format to a "flipped classroom", wherein students would meet physically, and I and the student assistants would "circulate" and discuss their projects with the students. I have wanted to try this for several years now, but we finally got access to a proper lecture hall in which to do so (R2). Without the proper room, the flipped classroom is very hard to pull off. Also, the pandemic forced us all to "up ouronline game". I used that opportunity to pre-record dozens of my own lectures (without an audience, as I prefer no audience to an audience of icons). Thus,students had access to all of my lectures, from day #1 of the course. They were encouraged to watch as many of them as they could, as quickly as possible, sothat they would have all the necessary background to begin their projects. Some students jumped at that opportunity and were well underway after only a weekor two of the semester, while others waited far too long to get started. I assume it was members of this latter group who, on the survey, suggested that I provide more strict timelines for the projects. I disagree with that view, since I feel that, by their 4th year, and in a project course, students need to be able to look at the specification of a project and come up with their own deadlines to get them to the finish line in time for my final deadline. The projects are described in great detail, and broken into many parts, so students have all that they need to begin any such planning process. In a 4th-year class, I am not going to hold their hands through this, though I certainly give hints along the way. Last year, it was mentioned that I should include more tutorials on some of the tools used in the course. That criticism was reiterated by at least one survey respondent this year as well. Although I agree and have tried to improve in that area, I gave that lower priority this year due to a new project (AI poker) that drained a huge amount of my time for preparing the 2023 version of the course. I implement all of the projects myself, before assigning them to the students, and this took a lot more time than expected for AI poker. In the process of this coding, I do my best to simplify very complex aspects (derived from complicated research papers) to a "digestible" but still interesting variant for the students. The goal is that, in the end, although I may have spent 2 or 3 months implementing something, writing lectures on background material, etc., the students will only need a fraction of that time given the simplifications and explanations that I provide. So that's the goal, at least: I do a lot of the worst "grunt" work and then provide a "kinder, gentler" version to the students, but one that retains the core concepts from the published research papers (on some of these ground-breaking AI systems such as AlphaGo and DeepStack). My other view on the tutorial-support issue is that there are already several good tutorials online. My web page includes pointers to many of them. For most advanced students in computer science, these tutorials are sufficient to get them started with numpy, or tensorflow, or pytorch or any of the other tools that they may find helpful in IT-3105. But since it is a very advanced course, I feel it "fair" to assume that students are capable of learning these tools on their own and do not need detailed assistance from me or my assistants, although I have never put limits on what my assistants should or should not help students with. I only put limits on what I feel are necessary lectures for me to give in the course; and tutorials on tools are always going to be considered "peripheral" in an advanced course such as IT-3105. To summarize, the switch to the flipped classroom is complete, and, judging by the responses to the survey, quite popular with the students. The call for moretutorials has not yet been met, but I will have it in mind when I prepare future versions of the course.

I want students to learn to tackle relatively large AI projects with sufficient but not excessive assistance. I'm going to tell them, in great detail, all the parts of the project and all of the code that needs to be written. But I will not give them many intermediate deadlines nor excessive constraints on exactly how their code should be structured. Although I do provide some "dos and don'ts" about code writing that I do enforce pretty strictly, I also give them freedom to explore many different software tools while offering some hints as to which ones I and/or my PhD assistants prefer. Given these hints and a large corpus of online lectures (around 10 hours per project) and associated "specification" documents (usually 10-20 pages per project), the students are "on their own". None are forced to come to the physical "flipped classroom" meetings, and only about 25% of the students do attend on any given week, but that has never bothered me. With a smaller group, it is easier for me to walk around and spend significant time with students who really want to talk to me. Sometimes, such a sit-down chat with a 3-person group will last 20-30 minutes. Since all of the lectures and associated materials are online, I can safely assume that all students have access to everything they need, so if they don't show up to physical meetings, they are either a) doing well and need no assistance, b) getting their assistance from the student aids, or c) not taking the course seriously, often to their own detriment. One very important question on my survey is this (with this year's answers included): Question 7:  Multiple ChoiceHow much practical information did you learn in this course, in terms of programming techniques and tools for use in artificial intelligence or computer science in general. This covers both things that were explicitly taught in the course and those that you picked up, on your own, as part of completing this course. Percent AnsweredVery little: 0.00% A bit less than what I have come to expect from a course in the computer-science department (IDI).3.225% About what I have come to expect from a course in the computer-science department (IDI).22.58% MORE than I have come to expect from a course in the computer-science department (IDI)54.838% MUCH MORE than I have come to expect from a course in the computer-science department (IDI)19.354% For this course, I consider this the most important question of them all. I am pleasantly surprised by the results this year, as I was last year. It should be noted that I only provided a limited amount of (what I would call) "practical" tips in the class. My lectures cut directly to the theory behind various techniques, such as Reinforcement Learning (RL). Although I do provide links to some code-writing tutorials - and some of my own documents on more practical matters - I definitely do not "spell out" all the coding details for these projects. I want the students to be forced to make those decisions, and many of them involve assessing the applicability of various online software packages for portions of each project. So I am very happy that 74% of the students feel that they have learned "more" or "much more" about these practical issues in IT-3105 than they do in many of their other courses. And since I simply do not provide all of this information, I assume that students were forced to "pick this up on their own" on their path to completing a project. This is what they will be expected to do in industry, so I am happy to see that IT-3105 is preparing them, in some small way, for the "real world", despite the heavy theoretical focus of my lectures.

This is a difficult course, both to teach and to take. Coming up with projects that give students a "good feel" for state-of-the-art research projects, but without letting them simply run downloaded code from those projects, has become a monumental task for me, as AI research has soared to such high levels. Before AI took off in popularity, there were many "famous" results that could be implemented by one person in a few days. Now, these projects often involve dozens of PhDs working many months or years to produce utterly amazing results, such as the AI systems that now routinely beat world-champion humans in chess, go, poker...or, of course, Chatbots that essentially pass the Turing Test. Once I get my head wrapped around these systems and manage to implement core aspects of them, the burden falls on the students to do the same, but ideally with less head-banging due to my (hopefully helpful) lectures. When this course began, under a different name, back around 2000, the goal was much less ambitious: teach fundamental coding techniques used in AI. Unfortunately, times changed, and there were fewer and fewer techniques and computer languages that were specific for AI. Most of them are used throughout the computer-science curriculum, just as AI has infused many other fields of computer science. I once emailed Peter Norvig, the co-author of the world's most-used AI textbook and the author of an earlier book entitled "AI Programming" -- Peter was also the research director at Google for a period. I asked Peter how he would teach a course in AI Programming now, and he basically reiterated what I just stated: there isn't a lot that's special to AI anymore, not at the low level of coding. LISP and PROLOG, the two key AI languages from the 20th century, are barely in use anymore. Almost everyone uses Python for AI, but Python is not exactly foreign to the rest of the computer-science department. So it's hard to carve out a niche for AI Programming. At the conceptual level, there are plenty of important AI algorithms, but turning them into working code is no longer much of an "AI problem". I am always on the lookout for coding techniques that seem particularly suited for AI but are not used much in other areas. If I can find them, I would like to change parts of IT-3105 to focus on them. But at the moment, the list is very short, so I will continue to focus almost exclusively on these 3-4 projects that I provide each year. From everything that I can ascertain from this survey and my private conversations with students and student assistants, the course is doing pretty well and is not in need of any immediate major changes. Finally, ChatGPT and CoPilot do worry me. The students that I have talked to (including my own daughter, who studies computer-science abroad), say that they use these tools as "an extra member of the team". They provide suggestions, just like everyone else, and then the group decides which suggestions to use. For these complicated projects, the AI tools cannot simply write all the code from a single prompt...at least not in 2023. But, a) the tools will improve, probably exponentially, and b) filtering suggestions is a far cry from actually GENERATING suggestions. Students learn (a lot) by going through the generative phase themselves, and the less they do of this, the more shallow their thought processes become. If you're going to understand the deeper aspects of anything, you need to grapple with the deep problems, not just select among proposed solutions. So I worry that a course like IT-3105, which is designed to make students write some pretty tricky code, FROM SCRATCH, is going to lose a lot of its value as students begin to use and RELY UPON AI tools to do most of the hard work. These same concerns surely apply to almost every department at NTNU. This is just my philosophy, but computer-science departments should be even MORE concerned, because I think that code writing (although perceived as "really complicated" by most people, myself included) has extremely high constraints, much more so than natural language. One mistake in a piece of code, and it simply will not run. Natural language, on the other hand, admits all kinds of variations, many of which are grammatically correct and thus perfectly "legal". But the text written by an AI system is often superficial and might only garner a "C" or "D" grade in, say, a college history course, since it wouldn't necessarily reflect many deep insights -- although we see exceptions all the time. In contrast, as long as an AI produces code that satisfies all the constraints (which are more restrictive than natural-language grammar), then it will run and produce results. It's then hard for a computer professor to give such code anything but an "A" or "B" grade. Although this may not seem logical to some people, it's pretty much certain that the MORE constraints that are inherent in data, the EASIER it will be for an AI system to generate similar, viable examples based on that data. Constraints make things easier for AI systems, and the data that we feed into a code-generating AI system is VERY constrained (structured) compared to the natural-language input given to these same systems. In short, this is just another example of what is often called "Moravec's Paradox" in AI: things are are relatively hard for us humans, like writing code, are EASIER for an AI system than things that are easy for us humans, like writing legal prose. So Chat systems may hurt computer science earlier than other departments, but no department is safe from this, only affected at a different rate and to a different degree.

NTNU:TIA:REST

Institutt for datateknologi og informatikk

Department of Computer Science

Engelsk

Mer om vurdering

Minst 5 av de 6 øvinger må være godkjent for bestått. I noen tilfeller blir to eller flere øvinger satt sammen til et prosjekt. Det betyr at en student kan stå en del av et prosjekt, men stryke en annen del.Gjentak krever nye øvingsinnleveringer.

Anbefalte forkunnskaper

Emnet bygger på TDT4120 Algoritmer og datastrukturer, TDT4136 Introduksjon til kunstig intelligens, TDT4171 Metoder i kunstig intelligens, og krever forkunnskap i Diskret matematikk tilsvarende MA0301 Elementær diskret matematikk.

Forkunnskapskrav

Emnet er valgbart for studenter som er tatt opp på studieretning i Kunstig intelligens innen Datateknologi (MTDT, MIDT), Informatikk (MIT/MSIT) og spesialiseringen Kunstig intelligens på teknologiretningen Datateknologi, Industriell økonomi og teknologiledelse.

Faglig innhold

Faget gir studenter anledning til å implementere mange av de klassiske kunstig intelligens (AI) algoritmene og så å bruke dem som moduler i større AI systemer rettet mot områder som lyd og bildebehandling, simulert fotball (f. eks. on-line Robocup konkurranser), poker-spilling (f. eks. Texas Hold Em) og robot navigering. Følgende AI algoritmer kan bli viktig moduler i disse systemer: A*, means-ends analyse, besluttningstre læring, genetisk algoritmer, neurale nettverk, bayesiansk klassifisering, case-basert resonnering, boosting og bagging. Gjennom det arbeidet får studenter en forståelse for AI i bruk i stedet for AI teoretisk sett eller AI på lekeproblemer, som man ofte får fra AI kursene på intro- og mellomnivå. Faget består av 2-4 prosjekter, avhengig av årets prosjekter og deres omfang. Hvert prosjekt skal støttes av et sett av forelesninger om relevante teoretiske og praktiske grunnlag, mens noen forelesninger blir dedikert til generelle diskusjoner mellom faglærer og studenter om et prosjekt og fremgang til de forskjellige student grupper. Studenter har fritt valg av programmeringsspåk, men Python, Java eller C++ vil ofte bli anbefalt.

Kursmateriell

Forelesningsnotater samt prosjektbeskrivelser skal deles ut for alle prosjektene. I noen sammenheng skal forskningsartikkler også bli gjort tilgjenglig. For robotikk prosjekter skal studenter få tilgang til en robot simulator og muligens til ekte roboter (for en begrenset period).Alt er gratis.

Læringsutbytte

Studenter får aktive erfaring i å designe og programmere rimelig store AI prosjekter. Studenter får verdifulle innsikt i hvorfor, hvordan og når man kan bruke AI metoder på ekte problemer som de kan treffe i arbeidslivet.

Læringsformer og aktiviteter

50% standard forelesninger, og 50% interaktive prosjekt diskusjoner mellom studenter og faglærer.Prosjektarbeid er individuelt eller i grupper på 2.

Arbeider 1

Arbeider 2

Arbeider 3

Arbeider 4

Arbeider 5

Arbeider 6

Kunstig intelligens programmering

Informatikk

Industriell økonomi og teknologiledelse - masterstudium (5-årig)

Datateknologi - masterstudium (5-årig)

Datateknologi - masterstudium (2-årig)

Informatikk - masterstudium

Master of Science in Informatics

Web/Lists(guid'8b698a2d-6ad0-4f7d-b3be-219cc1979f02')/Items(22494)/FieldValuesAsText

The main "action" here is to continue to check the literature for interesting research that I might be able to convert into projects for this class. That process is ongoing, and I think I've found an interesting new area or two that will serve as the basis for projects in 2023.

The goal of the course is for students to get good hands-on experience with AI projects of a significant size, i.e., not just "toy problems". Although some of the assignments are toy problems, to get the studentsfamiliar with particular techniques, about 50% of the class involves a very large project involving a simplified version of DeepMind's (now famous) AlphaGo system, which the students build from scratch and then apply to the game of HEX, which is challenging, but not as difficult as Go. The results of the course survey indicate that students were, by and large, happy with the class AND were challenged -- see the statistical summary of the survey results, in particular, question #3. Also, the answers to questions 5 and 6 indicate that students do get a solid mix of theory and practice in this course...and that is a central goal: I don't JUST want students to write a lot of code. I want that code to be grounded in reasonably advanced theory that the students need to absorb through my lectures. Regarding those lectures, I feel that their content is quite good, but the sound quality was not so good -- I got several complaints from students about that. I was able to "upgrade" the sound on those lectures that students complained about, but I fear that other lectures came out a bit below average, but that none of the students said anything about it. Personally, I had no trouble hearing myself when I replayed them, so I'm not sure if this is a problem with Panopto, the student machines or the original recordings. Next year, hopefully, the lectures will be in-person and these panopto files will just serve as backups or chances for students to listen several times. Since around 50% of the students responded to the survey, the results do seem meaningful. This course has received good feedback in the past, but this is the first time that I have gotten a solidly statistically significant set of responses. Using Blackboard for a survey seems like a much more useful approach than using reference groups, which do not give nearly this volume of feedback. Several files from the survey are included in this report. I'm sorry that I had to use screen shots for the statistical summaries, but I could not find any other way to get the information out of Blackboard other than the large spreadsheet, which is also included. Probably most useful is the file containing the "long responses" by students.

Some students request more of the practical information, whereas the lectures are purely theoretical. I have included a few practical lectures as well, but their quality is not that good, plus, I have found better lectures on similar practical topics online, so I have included links to those on the course web page. Many of the students are extremely self-sufficient when it comes to programming, so they figure these things out veryquickly; but others would appreciate a little more practical programming advice. I will try to rectify this situation in 2023. Part of the problem is that students are free to use many different software platforms forthese projects, so it's hard to give programming tips that apply to all of them. Still, some improvements can be made.

Karakter basert på prosjektdemonstrasjoner og rapport.Ved gjentak må hele emnet gjentas.

Emnet bygger på TDT4120 Algoritmer og datastrukturer, TDT4136 Introduksjon til kunstig intelligens, TDT4171 Metoder i kunstig intelligens, og krever forkunnskap i Diskret matematikk tilsvarende MA0301 Elementær diskret matematikk. 

Faget gir studenter anledning til å implementere mange av de klassiske kunstig intelligens (AI) algoritmene og så å bruke dem som moduler i større AI systemer rettet mot områder som lyd og bildebehandling, simulert fotball (f. eks. on-line Robocup konkurranser), poker-spilling (f. eks. Texas Hold Em) og robot navigering. Følgende AI algoritmer kan bli viktig moduler i disse systemer: A*, means-ends analyse, besluttningstre læring, genetisk algoritmer, neurale nettverk, bayesiansk klassifisering, case-basert resonnering, boosting og bagging. Gjennom det arbeidet får studenter en forståelse for AI i bruk i stedet for AI teoretisk sett eller AI på lekeproblemer, som man ofte får fra AI kursene på intro- og mellomnivå. Faget består av 2-4 prosjekter, avhengig av årets prosjekter og deres omfang. Hvert prosjekt skal støttes av et sett av forelesninger om relevante teoretiske og praktiske grunnlag, mens noen forelesninger blir dedikert til generelle diskusjoner mellom faglærer og studenter om et prosjekt og fremgang til de forskjellige student grupper. Studenter har fritt valg av programmeringsspåk, men Python, Java eller C++ vil ofte bli anbefalt.

Forelesningsnotater samt prosjektbeskrivelser skal deles ut for alle prosjektene. I noen sammenheng skal forskningsartikkler også bli gjort tilgjenglig.  For robotikk prosjekter skal studenter få tilgang til en robot simulator og muligens til ekte roboter (for en begrenset period). Alt er gratis.

Studenter får aktive erfaring i å designe og programmere rimelig store AI prosjekter. Studenter får verdifulle innsikt i hvorfor, hvordan og når man kan bruke AI metoder på ekte problemer som de kan treffe i arbeidslivet.

50% standard forelesninger, og 50% interaktive prosjekt diskusjoner mellom studenter og faglærer.Prosjektarbeid er individuelt eller i grupper på 2.

Web/Lists(guid'8b698a2d-6ad0-4f7d-b3be-219cc1979f02')/Items(17013)/FieldValuesAsText

Se vedlagt fil "it3105-instructor-report-2021.docx" for detaljer om hvordan det gikk i år.

<ol><li>Ja</li><li>Web-siden til faget samt Blackboard</li><li>Ja</li></ol>

<ol><li>Studenter lærer teori gjennom mye praksis, d.v.s. mye programmering av rimelig vanskelig teoretiske konsepter.</li><li>Jeg tror at det er høyt. De leverer gode projekter</li><li>Greit nok, tross Covid-19. Blackboard Collaborate funker ganske bra.</li></ol>

Karakter basert på prosjektdemonstrasjoner og rapport.
Ved gjentak må hele emnet gjentas.

Faget gir studenter anledning til å implementere mange av de klassiske kunstig intelligens (AI) algoritmene og så å bruke dem som moduler i større AI systemer rettet mot områder som lyd og bildebehandling, simulert fotball (f. eks. on-line Robocup konkurranser), poker-spilling (f. eks. Texas Hold Em) og robot navigering.  Følgende AI algoritmer kan bli viktig moduler i disse systemer: A*, means-ends analyse, besluttningstre læring, genetisk algoritmer, neurale nettverk, bayesiansk klassifisering, case-basert resonnering, boosting og bagging.  Gjennom det arbeidet får studenter en forståelse for AI i bruk i stedet for AI teoretisk sett eller AI på lekeproblemer, som man ofte får fra AI kursene på intro- og mellomnivå.

Faget består av 2-4 prosjekter, avhengig av årets prosjekter og deres omfang.  Hvert prosjekt skal støttes av et sett av forelesninger om relevante teoretiske og praktiske grunnlag, mens noen forelesninger blir dedikert til generelle diskusjoner mellom faglærer og studenter om et prosjekt og fremgang til de forskjellige student grupper.

Studenter har fritt valg av programmeringsspåk, men Python, Java eller C++ vil ofte bli anbefalt.

Forelesningsnotater samt prosjektbeskrivelser skal deles ut for alle prosjektene.  I noen sammenheng skal forskningsartikkler også bli gjort tilgjenglig.  

For robotikk prosjekter skal studenter få tilgang til en robot simulator og muligens til ekte roboter (for en begrenset period).

Alt er gratis.

50% standard forelesninger, og 50% interaktive prosjekt diskusjoner mellom studenter og faglærer.
Prosjektarbeid er individuelt eller i grupper på 2.



Studenter får aktive erfaring i å designe og programmere rimelig store AI prosjekter.

Studenter får verdifulle innsikt i hvorfor, hvordan og når man kan bruke AI metoder på ekte problemer som de kan treffe i arbeidslivet.

Web/Lists(guid'8b698a2d-6ad0-4f7d-b3be-219cc1979f02')/Items(11427)/FieldValuesAsText

<ul><li>Yes</li><li>web pages, lectures, course materials</li><li>yes</li></ul>

<ul><li>good match</li><li>variable, but mainly very good</li><li>quite good, I believe.</li></ul>

okPlease see attachments to get the full story behind the class. It's hard to describe things in the official format based on the official questions. In general, the class has been quite successful over the years. The students generally enjoy it, work hard, and learn quite a bit of relevant material. The review by the student reference group this year is particularly thorough and insightful, and I hope that my own document (also attached) sheds further light on the course and my assessment of it.

Emnet er valgbart for studenter som er tatt opp på studieretning i Kunstig intelligens innen Datateknologi (MTDT, MIDT), Informatikk (MIT) og spesialiseringen Kunstig intelligens på teknologiretningen Datateknologi, Industriell økonomi og teknologiledelse.

Web/Lists(guid'8b698a2d-6ad0-4f7d-b3be-219cc1979f02')/Items(4706)/FieldValuesAsText

Kunstig intelligens programmering - IT3105