The course really does a good job of illustrating that static and dynamic analysis can do a lot to ensure your software is error-proof. However, a lot of the methods are not super relevant in industry, which isn't necessarily a downside of everyone. The labs were pretty well organized but the lectures were hard to follow at times because the scope would change quickly.

I have been thinking a lot about how to review this course. It's a difficult review to write, because I think the organization of this course and the content is quite good, but there are several things that need to be improved.

The content is really quite interesting. I personally don't come from a computer science background, so I have never really dove into the details of what happens to go from writing words to making the computer do things. I really feel like the course team has put a substantial amount of effort into designing the course and improving it. Because of this, I feel like feedback/reviews are actually useful in this case.

The lectures are average to above average. Some of them are very long and jump around without logical flow. However, much of it is graphical and provide examples. They are solid.

The exam is fair. It's pretty conceptual, but so is the course. It's about as I expected. You can prepare well by following their preparation materials (again, good course team).

The labs. Oh man, the labs. This is where I really am torn for the review. The design of the labs is quite good. They really have the potential to be fascinating, and I love the fact that you have a sanity check with gradescope (again, good course team). However, on two labs in particular (1 and 5), I spent an insane amount of time on the silliest stuff. The reason for this is LLVM. The documentation for LLVM is terrible, and we really are not given sufficient guidance to implement the API's we're expected to implement. They do provide the LLVM primer document (trust me, use this). It's, by far, the most helpful document. However, it leaves out critical details. This was insanely frustrating. It is important to start early, because you will spend way too much time on things you shouldn't have to spend time on. It's just a game of guess, compile, attempt to decipher error. This leads me to the next review topic....

TA's and staff interaction. I have given them high praise, because they definitely have worked to design a good course. However, the teaching strategy of responding to every question with basically "can't tell you that, figure it out" is unbelievable. It feels like they leave these "easter eggs" of really annoying/insignificant things to figure out on purpose and then sit back and take great joy in watching you suffer. This is how I imagine the people that watch the cameras on escape rooms. As someone whose professional training is in education, I can tell you this strategy is not effective. There is a reason office hours are poorly attended: there is nothing to gain. They don't help you. I have reflected on how MY students would react if I responded to their questions in that way. It's completely unacceptable in my opinion.

Okay, bad stuff out of the way. It's a solid course. I learned a lot. I also wanted to bash my head against the wall on countless occasions. I recommend taking it.

This class is Software Analysis theory and LLVM wrapped in one. Course requirements and what is actually desired of you are two completely different things and it is quite asinine that they get away with this. They want you to be proficient in a C language, C++. That is in course requirement. No problem. The class opens up with a very rudimentary project about LLVM and to create some code that will iterate through LLVM's Module > Function > BasicBlock > Instruction hierarchy. This was challenging but not impossible nor too difficult. The quizzes accompanied by the EdLessons are also perfectly fine. So if you were unsure about this course and completed the first project thinking you could handle it, you're in for a surprise. Now you've passed the drop period and the second assignment all of a sudden wants you to understand LLVM in/out and use the various functions in its API without any explanation whatsoever. Documentation for linux software can be hit or miss, this one the latter, yet this is practically our only reference and many students shared the same sentiment that the docs were not sufficient. The course also points you to the doxygen, but does not teach you how to read it. Basically, the course assumes you are proficient in LLVM after the first project. If you think back to your "Introduction to Python" or "Introduction to C++" courses from your bachelor days, don't they go over code line by line and explain to you the syntax, operators, etc? Yes, they do. So given this is an LLVM introductory course, you would think they would do the same? No, they tell you here is getOrInsertFunction and llvm::CallInst::Create now figure out what to do with it. This is a course that we pay to learn but very few are learning. There also are no EdLessons for LLVM. Plenty for Software Analysis, none for LLVM. Cornell University has videos for LLVM for free, yet we pay money to attend this course and we don't get even one video. Should I have just gone to Cornell? Anyway, this seems to be another graduate level course where the instructors are too busy to instruct. Break the projects up. Make one project about figuring out how getOrInsertFunction works, then make the next project have us use that in a more advanced way. From the few courses I have already taken in the OMSCS program, I would think very little of this program if I were an employer.

Overall, I thoroughly enjoyed this course.

Throughout the semester, we were assigned concise coding exercises approximately every two weeks. These assignments were effectively structured and rarely posed significant frustrations. While there is room for enhancement in certain tasks, their shortcomings are by no means crucial. The experiential approach of engaging in hands-on activities truly facilitates comprehensive learning of each topic. The assessments, including exams and quizzes, maintain a balanced level of difficulty, not overwhelmingly challenging. Furthermore, the grading distribution is judiciously balanced and doesn't overly emphasize any single component.

With sufficient preparation, achieving an A grade in comparison to other courses is certainly feasible. The requisite time investment is reasonable and manageable. It's important to note that unless one is an expert in this particular field, there exists a wealth of valuable knowledge to acquire. As such, the course is not only approachable but also highly rewarding for those seeking to expand their expertise.

Course is designed well however, I felt that I did not learn much from this course. Exam was mostly conceptual and so were the quizzes. But the labs were pretty difficult and if you are not very well versed in C++, it is very rough. I had pretty good understanding of C++, I have used it extensively till few years ago, both at work and in competitive programming. In many labs, it was like try and check if it is passing and you do not know what you are doing and you want to do. At the end, I felt like I wasted one slot, which could have been utilized in better ways. Yes, TAs, peers were super responsive and extremely helpful.

One of the best organized and run course. I like the format with one exam and labs/quizzes.

Labs are the crown jewel of this course. All labs were implementing the content and reinforced learning. I applaud your decision to convert Lab6 to C++/LLVM. The deeper we can go in LLVM, better equipped we would be in that critical part of the modern toolchain.

I felt a midterm was needed to test theoretical underpinnings of the content. I will be ok if exam was deprioritized and more quizzes/labs were added.

The team has continuously improved the course and it shows.

Summer 2023 was not an option to select, but I took it current summer (still taking it). I have 20+ of software experience including extensive coding C/C++/Java/C#/SQL. Yes, I know a lot of people say it's an easy A, blah, blah, blah, but if you don't quite get the material from the lessons (after the first 5 times of rewatching it) then you are screwed, because there are no other resources online that explain it and the TA's are zero help (Me: I got 95% of the code working but this last part is not quite doing what I expected. TA: hmm, yes that should work, but I can't tell you anything else without giving away the answer. Me: Okay, thanks for your help /s). LLVM and the tools used in the class provide Doxygen documentation written by developers, meaning it is garbage. For most labs I found myself getting things mostly working then just throwing darts at the wall until something came back to pass the online grader. The labs themselves are all mostly theoretical and of no value beyond the class. Learned almost nothing from the class and yes it's easy compared to other courses (GIOS was much more difficult and I got an A without the curve) but don't expect to learn anything useful. I'll be lucky to get an A, will probably end up with a high B because the material and the class are outdated and simply terrible. I'm still waiting on the midterm exam grade which I have no idea how I did on it, maybe got 40%, maybe got 80%. The saving grace is the labs are 60% of the grade.

First, I would like to start by saying if you have experience with LLVM, not just passing knowledge of it, and you would love to explore more of LLVM and its uses for programming languages then this course is for you. The statement I have made is true for the first half of the course, the last half is where it becomes more Software Analysis focused and less Programming Language focused. With that said, the final half of the course is definitely my favorite part. This leads me to my second point however, the name of the course can be misleading and the same can be said for its course description. This course should be titled Compiler Middleware, Programming Language Quality Analysis, or Software Analysis w/ LLVM (something along those lines). Now that is out of the way lets describe the course. The meat of the course is the labs which there are 8 of. All of them, except lab 4, require extensive prep time due to resource gathering (finding forums on how to do x, y, z) because most of the time what is expected of you to do is straightforward but the resources for LLVM are unhelpful, limited, or are not in sync with the course’s version of LLVM. For example, if you want to inject a function call into a C program using LLVM during compile time, you have to call a function in LLVM with specific parameters (CallInst::Create(…)). One of the parameters is a class called Twine which is given a string in its constructor. The LLVM resource says that the Twine string is called “NameStr”, nothing else is alluded to what the purpose of the Twine is. Your first guess would be that the Twine is the name of the function you’re inserting into the C program because none of the other parameters seem likely, but you would be wrong. The Twine actually tells LLVM what the return type of the function you’re inserting into the C program is. This is what it’s like to engage with LLVM. The solution to these LLVM learning curves, in the course creators’ mind, is to provide a YouTube link talking about how to work with LLVM (not made by Georgia Tech) and the speaker having a busted microphone and provides basics about LLVM but not enough to help with any of the labs. Not to mention some of the resources found for LLVM to help are not applicable due to the course’s version difference. Admittedly, all of this becomes less of an issue by the time lab 3 (which is the 4th lab since the first lab is Lab 0) is completed because by then you would have learned enough about LLVM but it’s still quite vexing. This ultimately gives the course an uneven workload distribution. I distinctly remember spending ~30+ hours a week for the first half and then by the second half I was spending about 13 hours a week. If you have any questions, asking for help from the discussion board on Ed ranges from people providing pretentious/ambiguous answers or actually being helpful. However, being too helpful can get you in trouble which does explain the ambiguous answers. A great example of this was when I gave a peer of mine an answer on Ed by paraphrasing something I saw in the lab instructions. After I posted my answer, within an hour my answer was edited by an instructor to remove the paraphrase I used from the lab instruction pdf. When I approached a TA about why this happened even though I swore I paraphrased form the lab instructions, I was told that Ed doesn’t keep record of edits so he himself couldn’t prove it. All I was told was to be careful next time even though my answer to that peer was basically straight from the horse’s mouth. Speaking of TAs, their answers mainly lead to more questions. There are office hours where you can directly message a TA on slack but office hours open when the TA says they’ll take questions, not when its actually time to ask questions like the calendar says. When you do get help from a TA, the answers range from very helpful, answers that scream the-TA-doesn’t-know-what-is-being-asked and then they never get back with you, or the TA asking you questions that are about as annoying as “I don’t know you tell me”. In fact, I ran into a weird root privilege issue on my VM (the one provided by the course) that prevented me from starting one of my labs. I asked a TA what the issue could be so I sent them a screenshot. The TA said that what they could tell from the screenshot is bad and that I should get docker. I asked them what the problem with my VM is and they said “I don’t know” and that I should just get docker. I went through a couple of hoops to get docker running and then it just wouldn’t work. I tried to go back to my VM but virtualbox wouldn’t work because I reconfigured how my CPU did virtualization. Eventually, I found out my root privilege issue with my VM was a super easy fix and wasn’t a big issue after all but I ended up running in circles for a day or 2 because I got the wrong TA in office hours. In reality, there’s only about 4 good TAs out of the 10 available. Then there are the quizzes which are…ok. They’re fair but the quiz prep material prepares you for a quiz with light discrete math whereas the actual quiz is more “engineering” focused. One of the quizzes I failed because of a rule a professor never mentioned in his lecture (there’s no course text to check this either). Though these quizzes do help a lot in the midterm. Speaking of the midterm, it’s fair, gives you enough time, and is open book. And while we’re on the topic of good things about this course, there’s 2 midterm acknowledgment quizzes that get added to your overall midterm score which is neat. One other great thing is that Ed activity counts towards bonus points which caps at 5 percent points for your overall grade. Overall, I’m mixed about this course. The first half of this course was infuriating because of insufficient lack of help, guidance, and actual teaching of LLVM but come lab 4 the course becomes much easier, more interesting, and more enjoyable. Again, software being analyzed is the C programming language but if you love to analyze programming languages (specifically compilers like LLVM for the first half of this course) then this course is right for you. If I could give this course a rating of 2.5 and a difficulty of 3.5 then I would.

For perspective, this was my final class in the program. I do not write C/C++ regularly, but I did take CS6200: GIOS already.

No, this class is not about writing unit tests. This class is focused on static/dynamic code analysis. That said, I believe this is one of the most industry-relevant classes in the program and sorely underrated. Once you get past the LLVM learning curve, the class is very smooth sailing.

In general, the pace of the class is slow. In combination with the predictable grading, this would be a good class for a Summer semester or a double-up.

Grading

60% Labs, 20% Quizzes/Surveys, 20% Midterm Exam. Final grades were distributed on a standard 10-point grading scale. The class also offers 5% extra credit for participation. If you take the quizzes & exam seriously, it's quite easy to achieve the A grade.

Underrated feature of the class: almost all grades were published within 48 hours of the close of the assignment, including the exam grade.

Labs

We had 2 weeks to complete most labs with exceptions of Lab 0 (1 week), Lab 4 (1 week), and Lab 7 (3 weeks including Thanksgiving). This was way more than enough time. I achieved 100% on all labs without substantial effort: invest the necessary time to get the desired result and avoid obviously incorrect workarounds.

The class has a convenient Docker image for completing the labs. It's worth spending the time to get a good setup with automatic code completion. I used Docker + VSCode + Remote Containers + C/C++ extension.

The best resource was the LLVM_Primer.pdf document provided by the class. This contains the majority of the LLVM API needed to complete the labs.

The challenge is mostly connecting the lecture concept to the specific tools & implementation. All labs require <200 lines of code and some (Labs 4 & 7) require adding just a few lines.

All labs were weighted 8% with the exception of Lab 0 at 4%. All labs were opened during the 2nd week of classes, so students can (and will) work far ahead on the labs.

Lab 0: Intro to LLVM

This was a very gentle introduction to LLVM. The only goal is to loop through the hierarchy of LLVM functions/code blocks/instructions. You can effectively find the answer verbatim from the primer document.

Lab 1: Fuzzing (LLVM)

This was the first "real" lab and generally pretty tough while students are still acclimating to LLVM. Due to the random nature of fuzzing, I had to spend a decent amount of time tuning the randomness of my strategies. It's easy to underestimate the amount of changes expected between iterations.

Lab 2: Dataflow (LLVM)

I really enjoyed this lab. I had to spend some time to understand how the algorithm from the lectures mapped to the data structures provided in the lab. After figuring that out, this was quite easy & interesting.

Some students lost points from hardcoding a number of iterations for chaotic iteration. Don't do that.

Lab 3: Datalog (LLVM + Z3)

This was effectively Lab 2 but using the Z3 solver by adding constraints. Conceptually, this lab required a lot of time & experimentation to figure out. I somewhat brute-forced the solution by experimenting until the output worked.

Lab 4: Type Systems (TypeScript)

This was a pretty interesting lab: given some JavaScript code & failing unit tests, add type info & fix the bugs for 3 different programs. I use TypeScript professionally on a daily basis, so this lab was quite easy. The only challenge is fixing the issue in a correct way: there are many ways to pass the unit tests without using the intended values.

There was also some gray grading where the staff manually inspects the code for a sufficient amount of typing. In other words, you must have type annotations added to the functions or data structures relevant to the issues.

Lab 5: Cooperative Bug Isolation (LLVM)

In my opinion, this is the most conceptually & technically challenging lab of the class. Some of the LLVM instrumentation can be borrowed from Lab 1 but using additional API such as IRBuilder. The challenge increases in Part 2 with the implementation of the pseudocode algorithm from the lectures. The solution requires reading log files generated for each iteration, creating an algorithm with a feedback loop.

The lab is testing against randomized programs which makes solution verification much more difficult. I had to rely on Gradescope for most of the solution validation.

Lab 6: Delta Debugging (Java)

This lab was conceptually quite easy. Similar to Lab 2, most of the work was implementing an algorithm from pseudocode. This was a relatively rote lab after figuring out the stopping conditions & potential off-by-one issues. The only real Java knowledge required was a small amount of string manipulation.

Lab 7: KLEE (C + KLEE)

Similar to Lab 3, this involved using the KLEE tool to add constraints (klee_assume()) & assertions (klee_assert(0)) for the desired result.

I found this lab to be the most tedious. KLEE is not included in the base Docker image, so this lab required more setup. As a result, I had to work around path issues. I also wasted time before realizing that the "error case" produces the expected file at the absolute Linux path /tmp/<suspicious file>, not within the project directory itself. It's also hard to iterate quickly with KLEE because the output is a tool-specific format.

Students needed to constrain inputs to keep the runtime under 1 minute, but you could be penalized for excluding potential failure inputs. Apparently you're supposed to find the pattern of failing inputs through trial-and-error, but there's no good assurance to avoid over-constraining or under-constraining and there's no pre-deadline test cases on Gradescope to validate constraints. The staff acknowledged the blind grading situation but I doubt this can be easily addressed.

The staff discouraged this approach, but I ended up reverse-engineering obfuscated C code using ASCII codes & C standard library functions. This gave me way more confidence in my constraints. The "intended" approach seemed risk-prone for missing an edge-case.

Quizzes

30 minutes, unproctored, open-everything (sans peers). 1 quiz per lesson and you can complete the quizzes any time before the end of the course.

These can be somewhat tricky because there isn't much opportunity to practice for the questions. The teaching staff provided additional preparation questions & answers, but I found that the questions embedded in the lectures were often more relevant to the quizzes.

I lost trivial points on some quizzes because some questions ask for specific answer formatting. Take your time & reread the instructions before submitting.

Exam

3 hours, proctored, 24 questions. You could bring a page of notes, but you needed to fill your notes in a very specific Word document template & upload to Canvas so you could access the document without getting blocked by the proctoring software. Inconvenient, but the process of distilling notes to the specified format helped reinforce many concepts.

I felt the exam was tough but fair and only covers 4 lessons of material. Class average was somewhat low at 78%. You should expect to make up for this grade with high lab grades & extra credit participation.

My only advice is to carefully review all quizzes, lecture questions, & exam preparation questions. Copy useful diagrams (e.g. soundness vs. completeness) to your notes. You do need to know the details of all the specific tools mentioned in the lectures: Monkey, Korat, Cuzz, Randoop.

Background: I have been in tech for 4 years, graduated about 8 years ago with a Materials Engineering bachelor's, and was mostly self-taught in CS before this program (like 2 undergrad courses, some online courses, and small web dev bootcamp). I am currently working in data engineering, data analysis, business intelligence, etc. so these topics are not things I deal with on a day-to-day basis. But data engineering has been overlapping with software engineering for the past few years so I thought a course like this would be interesting and I didn't regret taking it!

My thoughts: This is my fourth course in the program and I've taken more "intro" courses like CS6400 and CS6310. I would probably put this in the same vein as those two where the concepts are not that complex and the workload is reasonable. I naively came into this thinking it was more about unit and integration tests but it was way more interesting than that on how to create way more effective testing. The course is really well run where the videos are directly related to the labs, the quizzes are challenging enough to teach you the concepts, the ed stem discussions usually have what you are looking for, and everything is available to you at the beginning of the course so I was able to finish the course about 4 weeks before the end of the semester.

I will say that LLVM was quite a doozy to get used to for Lab 1. I never took an assembly course nor have a deep knowledge of compilers and thought that would be an issue for the first few labs, but that wasn't hard to pick up on, I just had a hard time learning LLVM. So just anticipate that you will probably spend a good amount of time on that lab but after that, it's easier. I will say that the other reviews that describe how long each lab took are usually pretty accurate.

Software Analysis is my 4th class in the OMSCS program. It's been an awesome, well-run course--the best yet, so far IMHO. The lectures are concise and yet packed with useful information. The quizzes are good knowledge checks (although I will say the formatting of the form leaves room for goofy errors). The exam questions were straightforward but also not too simple. Projects were challenging, practical, and helped solidify the concepts covered in lectures.

My only regret is that I couldn't dedicate as much time to the course as I'd wanted. Personal matters outside of school are taking priority so I'm having to rush through most of the course content just to get it out of the way. Despite this, I think (and hope) I could still get by with a B.

Anyway, I'd recommend this course if you've got no other electives you're interested in or need a "lighter" course that's still useful relevant (in context of SWE/tech work). I would not recommend pairing this course with a "hard" course; especially if you're working full-time or have obligations at home that require your full attention.

This course is surprisingly good and I think it is underrated. It looks difficult at very beginning as LLVM looks like something complicated. However, after u can conquer the first two labs (Lab 0 and 1), it becomes a very easy course. Most of labs provide code frame that helps you to implement the functions. The assignments are well-stated. You def would learn something from this course.

TAs are not very helpful from my point of view as they only provide some vague instructions. Google and classmates are more helpful.

For background, this was my 7th course in OMSCS. I come from a STEM background (EE) and I have a STEM job (CS/IT).

I absolutely loved the way this course is run! The professor always started the lectures with an overview of what we were going to learn and ended with a very good summary. I personally love this format for lectures.

The lectures (material-wise) were very good and covered a wide range of topics for software analysis. This is NOT a software development course and this is NOT a software testing course. The lectures went over different static and dynamic analysis techniques that are useful when writing testing frameworks, IDEs and/or compilers. The course did not focus on one specific topic and sort of provided an overview of many different techniques.

The course had lecture quizzes (open book) for each lecture (due at the end of the semester, but I recommend doing them as you go through the lectures), one exam (mid term, decent difficulty, we got a 10 day window to do it) and 8 labs (lab0 to lab7 with Java, C, C++ and JavaScript/TypeScript). Majority of the grade is defined by how well you do on labs.

All future labs were made available right after the drop deadline so students could work ahead if they wanted to. I stuck with the regular schedule throughout the semester. The beginning was a little steep since I had to learn LLVM and brush up my cpp but things got much easier afterwards. The instructor team also provided good resources to get you started on LLVM and lab0 was very useful.

Hourly breakdown of labs for me was fairly similar to the ones provided by fellow students in the early reviews. Labs also complement the lectures, which is awesome. I wouldn't call any of the labs stressful or impossible. Some labs were more difficult than others but if you know the material well you can work through them.

All labs provided some sort of test harness so you could test things locally. All submissions were on gradescope with public test results immediately available. Instructor team runs some hidden test cases but those are more to make sure no one hardcoded their way through the assignment. Grade turn around was great too.

The course also has extra credit available for participation (i.e. being active on the forum, helping out other students, doing the course surveys etc.)

I did not need to go to any office hours, so cannot say much about them. Fellow students were all fairly active on Ed forum.

The labs are challenging (in a good way) but fair. They offer a great opportunity for the generous extra credit once you figure them out and help others. The instructor and TAs are very active on Ed and Slack. Unlike other classes, grade response is very quick. Today is the last day that assignments are due (8 AM), and we got our final grade at 2:45 PM.

Background

Graduated with a bachelor's in CS and have over 6 years of experience in the software industry with my day job being at a FAANG level tech company.

Pros

1. Really good course content. It gave me exposure to something new which I have never encountered so far as a software engineer in my day job. I've heard of some of the terms used in the course content but understanding them deeply truly helped.
2. The course is extremely well organized and structured.
   • At no point during the class I felt like I missed something.
   • The Ed Dicussion's pinned posts are always kept up to date on what needs to be worked on this week. Additionally, you can keep a tab on the canvas calendar for due dates and you'll be golden.
   • The grades are provided on time.
   • TAs and Chris (Instructor) are responsive on Ed on posts where their presence is required.
3. All the course content becomes available fairly early and you could choose to sprint and complete everything ahead of time. Only the midterm has a set time period when you need to take it, but it also gave us a generous 10 day period.
4. The midterm exam had a 20% weight and there is no final exam. You are also allowed to carry one page of your own notes while all lecture notes are provided as well. It couldn't be easier to give this exam.
5. A generous 5% extra credit for discussions and participation which is actually awarded quite fairly.
6. While you will be able to breeze through the content really quickly (if you choose to), you will be challenged enough to actually embrace the joy of completing the labs.

Cons

1. While you will get ample help from everyone over Ed, there is not enough collaboration or a sense of classroom when taking this course. It is more like do X, do Y and you're done. This could be due to a smaller batch size maybe but I feel like the instructors could try to improve this aspect of the course. Make Slack more lively maybe or have some more ice breakers in between.
2. Depending on your work, this course may not directly add value to your skillset. It has a very specific use case in the real world and you should evaluate the reasons for picking this course. For me, it was for curious learning and exposure to new things, hence made a lot of sense but it might not be for everybody.

Content

Quizzes

The quizzes are helpful to understand the concept and add a reason to put efforts in understanding the content well but sometimes the questions can be vague and not explained well in the lectures and you could lose points on them. However, it is not that big of a deal and there are very few such questions. Overall, I think I lost like 2 points out of 20 and I was easily able to make up for that loss in the 5% extra credit.

Exams

As mentioned above, there is only one mid-term exam which was super fair and quite easy considering you have access to almost all material. The time limit is 3 hours but I was able to complete it in half the time. The content covered in the exam was only 4 lessons that did not have an associated lab with them. The weightage of the exam is also very decent at 20% overall.

Labs

This is where the magic of the course happens. Most of the labs are based on LLVM instrumentation of code in C++. You can get by with a basic understanding of C++. I last wrote C++ about 10 years ago when I was in school and don't remember much now and have also never seen LLVM in the past, but I was able to get on with it quite easily. The lab material has ample guidance to get you started with everything properly and you will never feel that this should have been explained better or worded differently. I won't write a detailed note on each lab since other reviews already seem to have it covered pretty well but these labs were the most fun part for me. On completing them, I felt like I learnt something new every time.

Time Spent

I was evaluating my time sheet today to write this review and was super surprised by how long it took me to complete this course. So I took 12 weeks to complete all of the content and spent roughly ~50 hours in total to complete everything. This includes EVERYTHING! Setting up the environments, preparing for the exam, watching lectures etc. The only thing this time doesn't include is the time I spent on Ed for my extra credit and just helping out other folks.

Summary

The way I've written the review, it might sound like the course is a breeze but no it's not. You will have to spend the time to understand the fundamentals really well before you can apply them. But once you do understand them, you will be able to move really fast and have a ton of fun. For me, this was an incredible course and I had a lot of fun. Kudos to the instructors and team!

Background

I'm recently graduated with a bachelor's degree in Mathematics and Computer Science and I'm an Associate Software Engineer in a robotics company. This is my first OMSCS course.

Course Notes

For my full review, read this.

Exam

Pretty straightforward exam, I got 18.8/19.8. You are allowed to carry 1 page of notes so be sure to capture topics/key concepts that you're not confident about.

Labs

• Lab 0 - Warmup (1 hr)

  • Pretty basic lab, an intro into LLVM

• Lab 1 - Fuzzing (5 hrs)

  • I found some code online and decided it was okay to copy and paste it into my solution (it was always okay to do so in my undergrad college) but ended up getting a 0 since I violated the academic integrity

• Lab 2 - Dataflow (8 hrs)

  • Fairly simple, just had to implement an algorithm from the lessons

• Lab 3 - Datalog (3 hrs)

  • Super easy lab

• Lab 4 - Type Systems (8 hrs)

  • Super easy but most students end up overthinking it. It was fun using Typescript for a change.

• Lab 5 - CBI (13 hrs)

  • Related to Lab 1 so I ended up spending a little more time ramping up on stuff that should have been known already. Other than that, I don't see why people think this was a very difficut lab.

• Lab 6 - Delta (10 hrs)

  • Implement an algorithm from the lessons in Java. I'm unfamiliar with Java so it'll take me a little longer than usual but this was fairly simple to do.

• Lab 7 - KLEE (10 hrs)

  • This was a nice lab that built directly off of the lectures.

Quizzes

All quizzes were fairly simple once you pay attention to the lectures. I had to rewatch some lectures twice to understand certain concepts and algorithms. Quiz prep helped a lot as well. Make sure to do practice quizzes and you'll be fine.

Pros

• Very active TA's
• Well run course
• Great content
• Given all lectures and assignments early in the semester so you can work ahead
• Fair grading and exams
• I was able to finish this course with ~ a month to spare

Cons

• None

TL;DR

This class is well run and a lot of fun. I'd say it's a great first course to do but you also gain a lot of useful knowledge. Ramping up on LLVM can be a bit tough but a Primer document is provided for you which helped me a lot. Despite getting a 0 in Lab 1, I may still be able to get an A in this course.

Background:

Education: BS in Aerospace Engineering from top 5 school; lots of Matlab and Python

Experience: 1.5 years as a software engineer

Summary:

This is my first course in the program, and I must say, it was an amazing first course. I was hesitant going in because of my weak C/C++ background, but I couldn't be happier.

If you are going into the program and want a challenging (but not extremely difficult) course that teaches you how to program in low level languages and about compilers, take this course!

TAKE NOTE: This course is EXTREMELY front loaded. Keep this in mind when planning your courses! The first 3 projects (project 0-2) were pretty challenging; project 1 alone almost took me 30 hours to finish.

Pros:

• Extremely well run
• Fair grading
• Emphasis on applying knowledge and not memorizing material for exams (60% of grade is project based!)
• Ample extra credit opportunities (I got a bad grade on the exam, with my extra credit I'll still likely pull an A)
• The TypeScript project was really cool and helped hammer into my head why statically typed languages are super useful

Cons:

• Can be difficult if weak with low(er) level languages. As someone with a MATLAB and Python background from undergrad, it was definitely difficult at times
• Lectures can be dry at times; I just skip them now and read the powerpoints
• Wish there was at-least a couple topics on testing with regards to software engineering (such as unit testing)

This course is not very hard for me. It has 7 labs, 1 mid term exam and each module has quiz. Mid term exam is ok. Not very hard. I prepared it by review the slides. But for quiz, it is kind of annoying for me. Sometimes I misunderstood the questions and got low score.

• Lab 0: just an easy warm up. 20 mins spent.
• Lab 1: medium level and spent about 8 hrs.
• Lab 2: Follow the algorithms on the slides. Medium level. 6 hrs.
• Lab 3: Easier than medium level. 2 hrs maybe.
• Lab 4: Typescript. Easy level. 1hrs maybe.
• Lab 5: CBI. This makes me feel the most difficult. 12 hrs.
• Lab 6: Just follow the algorithms in slides. 2 hrs.
• Lab 7: This one is the second difficult for me. I have been frustrated for a super long time and no idea. At the end, I also was not able to get the full marks. Tried my best. Spent like 12 hrs.

This course was primarily a survey of various forms of automated software testing (eg. test generation) and static software analysis (eg. type systems and data flow analysis). The course doesn't go deep into any of its topics, in fact most of the coverage is extremely shallow, but it does cover a pretty broad range of techniques.

The course has 1 exam, which is early in the semester, and 7 labs. The labs are due throughout the semester, but are all available after the drop deadline. There are also quizzes that accompany each lecture, those these are quite easy and open note. The course is so front-loaded that I was able to finish in about 8 weeks (though I have amortized my reported workload across 15 weeks).

The course is very well run and organized, the TAs are responsive and quick to grade, and the lectures are clear and straightforward.

My only complaint about the course is that it is pretty superficial. The lectures aren't very rigorous, and most of the labs are pretty easy. For the most part, they are of the "fill in a few methods" type, where the methods are algorithms straight from the lectures. Like others mention, many of the labs use LLVM, which can be fairly tricky. Don't let that fool you into thinking this class will teach you enough LLVM to be good at LLVM - the labs require only a tiny portion of LLVM features, and much of the scaffolding is provided in the starter code.

As promised course does go over software analysis & testing tools and techniques. However I felt the way it is delivered is not effective.

Cons: Lecture videos were done way back by the professor who is no more working at GATech. Lectures are bit dry and feels boring at times even though they are clear enough. Course is mostly around LLVM. If you do not get hang of LLVM then you are pretty much guaranteed to do worse. Plenty of labs but help is almost zero so you need to figure out on your own. Office hours are plenty but don't expect much help. Even if you are looking for help to understand concepts, felt not much help from TAs. Also beware of hidden lab test cases as they can surprise you after grading is done. During office hours, most of the time TAs would refer you to look at lectures (I feel understanding concepts from lectures are part of the problem at least for me) Mostly Ed discussion forum is I feel mostly to show off by the students to get extra 4% but not much help for an average student genuinely looking to understand concepts. I felt that students does not want to provide more details to questions posted mostly with the fear that they will get penalized. Very stringent grading. As mentioned in one of the Ed posts, even if you get 89.95 you will still get B (no rounding off to get A).

Pros: Well run course by TAs with plenty of office hours. Even though they did not work well for me, it might be opportunity for some students to ask right questions to get some direction on labs. Grading is very quick. If you are good student who can manage well, you have an opportunity to finish all labs in advance so you can use your time on something else. If you are interested on understanding some insights on compilers this course will definitely provide that using LLVM. Nice overview on different tools and techniques exists to uncover software bugs.

This was my second course and rather liked it. Take this course you enjoyed studying logic and proofs. It gives you a decent, practical insight into a few different analysis approaches and some design patterns that could be applied to new kinds of code analysis.

There's one lecture each week, followed by a quiz on the same content. Most weeks there's a lab. The first ones were the hardest, where you need to instrument code during compile time. Most of the trouble was just getting to grips with what is happening and the LLVM APIs. Some of the labs are much simpler, like finding three errors in some typescript. There's also an exam, but it only covers four lectures and understanding the lecture material is sufficient.

Apparently the summer semester is rushed, but the pace felt fine to me.

Difficulty:

• Most of the lesson content is only tested via quizzes, which are open-notes and can be taken immediately after watching the videos. This really helps keep the stress level low. Some of the quizzes do require very careful attention to detail, though, so don't take them lightly.
• There is an exam which covers a few lessons' content. I think this was more difficult than the exams in Networks or GIOS, and most of the points I missed in the class were on this exam.
• Most of the labs involve C++, but the amount and complexity of code required is far less than for GIOS. Most of them only took me one or two sittings.
• The labs are the biggest portion of the grade. All except one had autograders that you could make unlimited submissions to. They don't tell you the results for some test cases until after the deadline, but it seems like they tried to make those hidden test cases very similar to the visible test cases. So if your submission passes the autograder, you're probably going to get 100% - or at least, that was my experience.

Feelings:

• The labs are interesting and go just deep enough to give you a real feel for the concepts and tools without becoming tedious. The only one I disliked was the one on TypeScript - it won't teach you anything new if you've done any significant amount of work in a statically typed language already.
• The lecture videos are clear and to-the-point.
• I don't expect to be able to apply any of this in the near future, but it was fun to learn about. Statistical debugging does seem like it should be practical and highly useful in many contexts. Dynamic symbolic execution also seems like it could be very powerful in certain situations, so it was neat to be exposed to the tools.

Workload: I had to maintain focus and produce work each week in order to complete the course during summer semester. Quizzes kept me busy every single week. Content: This course provides different view points on how bugs affect the program and ways to reveal them. As it turned out, similarly to other computer science problems, testing has its own difficult challenges, especially as app size grows. Projects: Similarly to other OMSCS courses, be ready to dig deep recursively into some very narrow libs and solutions in C++ and LLVM IR. This course has a lot of compile level stuff, some assembly language (IR), multiple labs.
Outcome: After completion, I feel that I can discuss relevant topics in the field, understand issues related to producing a quality code and consequences of having bugs in a program, understand testing approaches and their limitations.

Really interesting course and a lit of fun. Overall the projects are straightforward and interesting to work on. The hardest part of the course is the beginning, so if you can make it through the first couple of projects the rest is pretty much downhill from there. One suggestions would be to brush up on your C++ and learn LLVM before the course if you have time.

This was my first OMSCS course and it was a good one! The topic itself was not one I had much interest in, but it turned out to be pretty interesting. The assignments (labs) were very technical and just challenging enough. I hadn't touched C++ in at least 5 years but there was no real in-depth language knowledge required to complete them. My lowest score was on the exam but it was easy to make up for that with the labs and get an A in the end. The profs and TAs are very responsive.

The overall class has good topics to learn, and I enjoyed specific labs, especially the last two. I was not too fond of the LLVM labs and felt like the whole class is around LLVM. I was waiting to get out of those labs. Finally, the last two didn't have to do with LLVM. If you plan to work as a developer, this class could help you improve your code and analyze it better, but if you are going to work as QA, you will never use any of this topic in real life. I felt like that lecture does not prepare you enough for the labs, and you will end up doing a lot of outside research to understand what needs to be done. Lab write ups are a little confusing and don't tell you precisely what needs to be done. Student can understand one thing and deliver that and later on finds out there understanding was wrong and get bad grades(Especially Lab 7 - KLEE). To avoid that, I will recommend starting ahead of time and utilize Piazza posts and confirm what you understood to do is correct. You have a chance to finish one exam in the middle of the semester(not comprehensive, only covers 4 chapters !!!), and if you finish the rest of the labs right away, you can clear out the rest of the semester and use that time for other classes or take a vacation, up to you. Instructor Chris is active on Piazza and does help out all the students, I had some conflict during the end, but overall he is an excellent instructor. Just the course "creator" and "organizer" needs to find a better way to organize course lessons and create contents in a way to prepare students for the lab well and not just throw them under the bus. You will spend a lot of time trying to understand the lab material from the links provided to understand it better. Overall, class is not hard at all, and you can get 5% extra credit for just participating on Piazza. If you still fall short by few points to make an A or B in the class, don't even bother to message them as they won't give you extra points as "it won't be fair to other students".

To sum up, I really enjoyed this course. I came in knowing nothing of software analysis as my undergrad didn't have anything like it available. Overall, the class isn't useful for my professional life but I imagine it's good to know other aspects of software engineering that I obviously wouldn't see. I thought it was neat learning that clang is based off of the compiler, LLVM, that you work on for the majority of the class. Also, this class is basically about software analysis and the correctness of them (not unit testing or whatever with devop unfortunately).

There are 7 labs, 11 quizzes, and 1 exam. The labs with LLVM (so 3 ish labs) were rough because the compiler doesn't exactly have the best UI experience to find what you need. I think every other lab was pretty fair and interesting in their own right. The majority of the work is in C/C++ and the other 2 labs in Java and Typescript/Javascript.

The quizzes and exam were rough for me because there aren't many materials you can find outside of the coursework. Any other resources from other universities you find they come from this class and perhaps the professor haha.

Anyways, the communication platforms were great as well. The TAs respond pretty quick on piazza but not as much on slack besides office hours. Slack is mainly driven by the other students. However, the class does gets pretty quiet second half of the courses as I imagine a ton of students have completed the rest of the assignments (as soon as registration is over, all the assignments and quizzes are open to be completed as early as possible).

There is extra credit which is 5% when I took it.

I took this course thinking this would be easy and will focus mainly on testing strategies/principles and so on. In the first two weeks, I realized what I have signed up for :). Programming is not an issue for me but definitely C/C++ is not my choice of programming language. I have extensive experience in java and more recently in Python by working on some other courses. Labs were challenging and interesting. They were based on concepts covered in the lecture. Based on the experience I have with some other courses, I do not enjoy going through lectures very much. I do not like 5-6 minutes on each page video. In this course each video was short and to the point, ranging from 1-2 minutes. While working on the labs, there were times when I was stuck for 2-3 days and all of a sudden a single line of code change solved the issue. LLVM/C/C++ were all I could say new to me but I love programming so I made it through.

Two of the main factors which helped me crack all the labs:

1. Extensive help/discussions on piazza where instructors provided enough hints, and students shared their experiences, sample test cases, and so on.

2. LLVM primer document - Many things that were needed in the lab were mentioned in this document.

There were 8 labs. One was in TypeScript and one in Java. Rest all were C/C++ LLVM based.

Thanks.

This is a Goldilocks class on every metric: not too X, not too Y, just right. It's a survey course in the best sense; you'll go just deep enough to learn the fundamentals on a number of different interesting subjects important to our field, each via an engaging, but not overly difficult, lab, yet have the opportunity to dive much deeper if you'd like. The instruction team is awesome to a person -- this is literally the first OMSCS class I've taken where I had nothing but positive, informative interactions with every Instructor and TA -- extremely well-organized, and lightning fast with the grading turnaround. Also, everything is released very early on, so you can work ahead; I was done about a month and a half before end of semester. And there's ample opportunity to earn extra credit simply by adding intelligibly to the conversation on Piazza. Great class!

I highly recommend this course.

This course is about using different techniques and tools to analyze and test software. It is NOT about writing unit tests or testing strategies but rather using tools to find bugs and assert correctness of a program in an automated miner.

For this semester we had a quiz after every module, 11 quizzes total. You have 30 minutes to finish the quiz but this was overkill. All but a few could be done in a few minutes, with the harder ones requiring you to run an iteration or two of some algorithm which can take around 10 minutes for most people.

There was one midterm and no final, with the midterm covering 3 modules. The midterm was virtually proctored but open course slides (then ones provided by canvas). Additionally we could upload one page of notes.

There were 7 labs, with one being "lab 0" worth a lesser piece of your final grade. Five of the labs where in C++, one in TypeScript/JavaScript, and one in Java. The reason why I marked this course as medium difficulty instead of easy is a few of these labs require you to use a library called LLVM. If you don't have familiarity with this library or are skilled at digging into documentation that doesn't have something like a "quick start guide" then this will cause a jump in difficulty. For example the "lab 0" was basically just a primer on LLVM. If you knew which library calls to make it would take minutes to complete, but took me upwards of 5-6 hours.

In all the courses I've taken from OMS this course made me feel like the instructors wanted me to succeed the most. Most of the information, LLVM specifics aside, was very clearly given with none of the usual "the proof is left as an exercise to the reader" kind of thing some other courses do.

Overview

I thought the class was pretty interesting. Its hard for my scatter-brain to put thoughts on paper about its usefulness. It seems like I might apply the concepts I learned in my career. It did not blow my mind like other reviewers. I didn't get on slack or office hours once. I pretty much only used Piazza to get extra credit points, and only looked at a few posts closely. But hey those extra credit points are super helpful. I got a 32% on one of the labs but was still able to get a good grade (maybe an A? the semester isn't over yet) in the class.

Quizzes

Open notes, internet. You have 30 minutes to take them and they are really short. I got 100% on almost all of them. Just go through the example questions and you should be fine. For the most difficult modules I watched the videos closely taking notes and spent 2 hours going over example problems before quiz. For the easier modules, I barely listened to the lectures and spent 5 minutes preparing for the quiz.

Exam

No surprises. Just go through the lectures and notes and practice problems. I studied pretty hard, I spent some weeknights and almost an entire Saturday studying. I got an 83. But I didn't feel that bad about it. Its not that much of the grade.

Projects:

Lab 0: Intro to LLVM

Workload: 9 hours. Really easy as far as technical complexity. But it took me a good while just scrolling through stack overflow and trying random things to figure out the LLVM library. I would recommend finding some intro to LLVM exercise somewhere online to prepare yourself (but if you're like me, you'll just dive in and see what happens).

Lab 1: Fuzzing

Workload: 12 hours. I actually got a zero on this assignment because I didn't follow the directions. (But then got it regraded for a final score of 32%). Make sure that you ONLY CHANGE THE FILES THAT YOU TURN IN. If you change a file in the repo that is not turned in, then when they try to compile with the unchanged file, you will have compilation errors and get a zero on the assignment. Even with a zero I MIGHT squeak an A on the class because of extra credit. If not an A, I'll get a high B.

But anyways, this assignment was not that bad. Just a lot of reading the same stack overflow posts 5 times for me.

Lab 2: Dataflow

Workload: 18 hours. I found this assignment to be a ton of fun. I had to come up with a fairly complex algorithm totally from my head. I have hit graph algorithms a million times in various comp sci classes, but this was my first time really making up an algorithm to suit a purpose. It was hard and I had to seriously engineer. I tried an approach for hours that ultimately didn't work. I loved it, though. I can imagine this project being next to impossible for someone with no C experience and no engineering experience and no graph algorithm (binary trees, red-black trees, blah blah blah, all that stuff I took in undergrad) experience. But if you want to actually get something out of this degree, I greatly recommend this assignment. This is pretty much the whole reason I liked the class. It was awesome when I got that algorithm running. You really have to actually dive in and think of novel ideas to engineer something to work. This is like real grad-level work, just designing an algorithm to suit a complex task with no help. I am sure you could find this exact algorithm on the internet to help you do the assignment, but the teachers warned that if you even look at another implementation its plagiarism. So I just engineered it from the ground up.

Lab 3: Datalog

Workload: 4 hours. Just basically clicking some things into place so that a third party algorithm can solve the problem for you. Its basically taking the algorithm from lab 2 and using a third party generic solver to do it for you. Its really cool that this technology is available, so I find this a valuable assignment. It took me a few hours to get the little mathematical equation type stuff formulated correctly. This is what you pass to the third party solver for it to do your work for you.

Lab 4: Type Systems

Workload: 1.5 hours. Super easy. I spent a few extra hours just researching type systems for my own purposes. Honestly, it was still kinda cool. But yeah, I finished this before 10am in the morning.

Lab 5: Cooperative Bug Isolation

Workload: 12 hours. My scatterbrain seems to have already forgotten this lab. It was pretty interesting content. It teaches you what is happening when microsoft asks you if you want to send usage reports for its software. I barely remember doing the assignment for some reason, but remember it being somewhat easy because I have gotten used to LLVM at this point. Still took a little while though. Don't expect this class to be an absolute breeze by any means. Even relatively easy stuff still took me a good bit of elbow grease.

Lab 6: Delta

Workload: 13 hours. I found this project to be kinda like Lab 2. You had to just implement the algorithm yourself. Although it was basically explained you in its entirety. So you didn't have to come up with super novel approaches yourself, because the algorithm was pretty cleanly explained for you. I enjoyed the assignment.

Lab 7: KLEE

Workload: 2.5 Hours. I was able to work ahead of schedule because some of the assignments were so easy (labs 3,4). So I have 4 Saturdays to do this assignment. KLEE seems really cool! The assignment is way too easy. I basically did it just by fiddling around with the code. This is the first semester in which they are offering this lab, so I suspect it will never be this easy again.

Final Thoughts

Overall, I find the class to be pretty fun. I like being challenged and felt like the instructors did a pretty good job. I think it is very likely that many of these topics will come up in my career at some point. I would recommend this class as a fairly easy option (for someone with a CS undergrad and work experience). It gets way easier at the end of the semester. If you could somehow pair this class with a class that gets hard at the end of the semester, you would have a perfect set-up to get 2 classes in.

This course is amazing and should be mandatory for any software developer in my humble opinion. It definitely changed my thought process about writing code and testing, thinking thoroughly about the variables/pointers that are being created in memory, making sure to free those resources. Selecting the right tool for testing and finding bugs is also very important. This course helps you do that.

I do agree that you might have a higher workload per week if you are not familiar with C languages like Java, C#, C, or C++.

The labs are very useful to understand the concept in the lesson. The primer offered at the beginning is very useful to get you started with some C++ and everything you need to be successful in this course.

This is my first class in the program and I like it!

Pros:

• I learned basics of C++ (if you don't know this already and don't have a strong programming background, you're going to have a bad time...)
• Only one midterm exam and tbh it's pretty easy. Don't expect 100% because it's short and you don't get scratch paper. I got an 89% after ~1 hour of extra review.
• TAs and students seem engaged on Piazza. Usually my posts got responses within 1 day
• Office hours are empty except for the weekend before labs are due, so you can definitely get 1:1 help

Cons:

• Some labs are WAY harder than others. I feel like a lot of the difficulty is front-loaded because you need to learn how to use LLVM libraries. The class uses clang/LLVM 8 but most docs/resources online target v12, which bit me a few times.
• The class VM is awful. I ended up installing clang/llvm 8 on my host machine and double-checking my lab on the VM after, which worked great. This also lets you use CLion as an IDE, which IMO is a must!
• Some TAs are way more knowledgable than others. I got bad advice from two different TAs that burned a few hours of my time. The head TAs are extremely knowledgable and give great advice, so strategize which OH you go to.
• Clarifications to the project often get buried in Piazza where you might not see them. I was frustrated with how the TAs handled clarifications.
• It feels like starting the projects early is a disadvantage. Everyone starts posting hints/discussions the Thu/Fri before the Mon when the lab is due and that info can save you a ton of time versus struggling on your own.

Extra:

• The lectures are very boring, but not as bad as others have said. I'll take dull and clear over energetic and confusing any day.

Overall, not a super hard class if you have some programming/OS experience. Most of my effort was ramping up on C++ and LLVM.

This class is very focused on different techniques for performing software code analysis. The material can be a little dense, and watching the videos more than once is very helpful.

There were seven labs, and each one tied back to a specific video lecture. Each video lecture also has a corresponding quiz. I found it better to watch the video, start the lab, watch the video again, complete the lab and then take the quiz. The quizzes all have a due date, but you can turn them in late up until the last Monday of class with no deduction.

There was one exam this term, a mid-term exam. The exam was tough but fair. The one downside is no paper/pencil is allowed and for someone like myself who likes to work ideas out on paper, that made the exam more difficult than it needed to be. I ended up getting a C on the exam, but an A for the class as I did well on the labs and quizzes as well as regularly attended at least one office hour per week.

The bulk of the coding on the labs is in C++ but you can do it in fairly straight C or just google to find the C++ way of doing it. There was a nice C++ primer at the start of the course and it really did cover everything you needed to know about C++ to complete the labs. In addition, there is a starter lab in LLVM which is used in several subsequent labs that was helpful in getting familiar with the LLVM library.

Overall this is a well put together class. All of the labs are open to start at the beginning of the class so you can set your own pace as you like. The TAs are available to answer questions either via Piazza or on the slack channel during the posted office hours.

I came into the class with a strong C/C++ background and several years of professional experience. Software Analysis was not something I had ever really considered and I will say that I learned quite a bit.

If you are someone who likes solving puzzles and finding bugs, this class will probably be enjoyable. If not, then you may want to look elsewhere.

I think this would be a good first course for many entering the program, as you really just need a pretty basic programming background for the labs. While I found several challenging, the complexity comes mostly from the LLVM infrastructure you work with in most labs I thought the course lecture content was interesting, although I also feel like it may be material I won't utilize much in my professional career. If you are interested in automated testing then this is definitely a must take. TAs were good, quizzes and the one exam seemed fair

Teaching team is great. The lectures are well done and easy to follow, with lots of examples given. Grade turn around time on the projects was very fast, usually just a few days. There were seven projects, with varying difficulty. Some were easy and some very hard. Decent variety in the projects. Several are using LLVM/C++, but also a few with different frameworks which provides variety. The teaching team provided a C++ primer, non-graded exercise to help students without C++ experience get their feet wet, which was very helpful to me.

Additionally, the projects are all unlocked early, so you can work ahead if you want which is good for flexibility.

Only one exam (midterm), which was challenging but fair and allows for one page of notes.

You will use C++ and LLVM for 4 of the labs in this class. The first lab gives you a gentle introduction, then the other labs get a bit more in depth. A primer, with relevant LLVM functions and code is shared, which greatly helps with the first few LLVM labs. Many labs use C++, with two labs that use TypeScript (like JavaScript) and Java. They have an optional primer for C++. I didn't do it, but it may be helpful. You should understand basics of C++, including use of pointers / references, standard libraries, and namespaces.

TA's are helpful. Some of the TA's often went out of their way during the Office Hours, spending time with each person, often going longer than the 1 hour time in order to communicate, understand, and help each student. I have to be honest, though, a few times in the Office Hours Slack chat, a TA told me something that I later discovered to be wrong or which lead me in a direction that was away from solving the problem. In retrospect, though, I did learn much more about the topic by spending extra time going through the problem.

Slack is used for Office Hours for one hour sessions most days of the week (twice on a couple days). TA's did the best they could, but they were often stretched to the limit with the number and depth of problems to assist with.

A positive for this class is that grading turnaround is fast, most labs grades were in within 3-4 days. You quickly know where you stand, shortly after an assignment deadline is met.

Another positive is that many labs are opened early, so you can start working early. Procrastinating does give more feedback and guidance from Piazza, but you probably learn more by figuring those things out on your own.

The midterm test was tough, but it was very well written and on topic. It's a big chunk of your grade, but if you do well on the quizzes and labs and participate well (there are points for Piazza participation, which are quite generous), you should be covered.

I enjoyed this class, and would recommend it, even though at times it was quite frustrating. I don't have a degree in CS, but I've done a lot of self-learning / studying and work hard to prepare in advance.

Like any class, you basically just need to read the manual, read the docs and primers, check Piazza and Slack, and check the docs for LLVM (or Datalog, which also presents a bit of challenge).

My hours per week are high, but that's because I spent a lot of time working ahead, or a couple weeks where I was stuck on a project concept. I also like to do extra stuff like create test cases or scripts and answer questions on Piazza. You can certainly get by with many fewer hours, but like most learning, you get out what you put into it.

Coursework

For the most part I felt like the assignments for the course were pretty good. They do a good job of actually getting you to understand the major concepts of the class.

I will say that the major exception to this is that the content of the assignments - tests, quizzes, and assignments - is frequently beyond the scope of the instruction/videos. It's extremely common for new syntax which was never used during the videos to be introduced without explanation during assignments, and variables are also frequently used without any explanation as to what they mean/reference.

Instruction

The videos for this class are by far the most monotonous and boring that I have seen in my academic career. TAs never give any sort of in-depth feedback or advice, and I suspect quite strongly that this is because they don't have a solid understanding of the course material. They've frequently been unhelpful even with very pointed questions on certain types of analysis, and are never able to generalize or rephrase concepts that I asked for additional explanations on - instead just pointing to the modules.

Workload

This course is pretty trivial if you've ever taken a class on testing or compiler design before, or have industry experience with automated testing and the ability to generalize and think rationally about it. The 'recommended readings' for the course are absolutely insane, however. For any given week it's not uncommon to have 3-4 academic papers linked, an external textbook with 1-5 chapters recommended (there is no official text for the course), as well as the entirety of documentation for some esoteric tool that no one outside of academia actually uses - about 1-3% of which will actually be useful/used, but no direction beforehand as to what part that will be.

Overall

Overall I wouldn't take this class unless you have a deep interest in testing - specifically the development of automated testing. Most of the techniques introduced in the class are low-level things you would expect at either the compiler or linter level of "testing", nothing like e2e or even unit tests. As a result, the class offers little that will prove useful in industry, and instead spends a huge amount of time on esoteric theory which is well established (though poorly explained and with little standardization which makes cross referencing to other sources difficult) but rarely applied outside of language design.

I'm halfway through this course and I would suggest you NOT take this one if you don't have cs background before. I have no idea what the class is talking and I don't think the content will be beneficial for a career in cs. You could hardly get any help from TA because they won't tell you anything. This is desperate.

Didn't really know what to expect from this course but I wanted to get exposed to this domain. I think this course does a great job in introducing a beginner to this landscape. I would love to see a sequel to this course that would select a couple of tools from this course and have students apply that on open source projects

Taking this class in the summer was a bit of a challenge since they crammed 90% of the material from the full semester into the shorter term. There was only a single midterm exam and no final which I liked very much. A large majority of the grade is projects which are due almost every week. I would say definitely go through the optional C primer at the beginning of the class if you do not have any previous C experience as it will make your life easier. I would say a some previous exposure to C is really helpful.

Overall I thought this class was executed very well with the TAs and instructor being accessible via Piazza. And were very accommodating to life events which was showcased by their Covid response allowing extensions and options. I found the material to be a bit dry but that is purely personal preference as this class was very well organized and gave a very good broad overview of software test.

This is a great course on how to analyze code and applications for bugs. It sounds like testing, but there is much more emphasis on pre-testing code with instrumentation. It is very insightful. It also uses a ton of different languages (CPP, Java, Javascript/Typescript) which is pretty cool if you are interested in working outside your comfort zone.

If there is one thing to know: LEARN LLVM BEFOREHAND The first few labs are based on llvm instrumentation and the learning curve was by far the hardest part of this course for me. Just run through some tutorials on it and try to read through the docs (because they were very difficult to traverse for me initially, then I got the hang of how they worked...)

One of the most well-run course in the entire program. Fantastic team of TA who genuinely cares about the students. Very fast turn-around for grades too. The material was a little dry though. Overall, an OK class.

I took this course for breadth, as I had no SWE training or experience, and had heard good things about this course from friends. This class is called "Software Analysis" (not really "Software Analysis and Testing"), and I didn't really know what to expect going into it.

Content

Software Analysis is about...analyzing software. Therefore, it's not just about "how to write tests for software" - it's about all kinds of different ways you can analyze software, and what the results of the analysis can tell you (soundness vs. completeness/false positives vs. false negatives).

The content is frequently abstract, math-y, and theoretical, although there are examples here or there about how useful some of the techniques are. I found the explanations to be clear and thorough, although not necessarily engaging. I didn't expect the content to be as abstract as it was, but now that I know more about the topic, I think that's appropriate.

The lectures were a little long but not oppressively. They're all on Udacity. There were generally several little quizzes sprinkled throughout each lecture that I found helpful.

Projects

There are 7/8 projects (there was a "project 0" - depends on if you count it). The first 3/4 projects are the hardest - 0-2 are hard because of LLVM and C, and 3 is hard because of Datalog. The rest of them take far less time (but it can be harder to get that "warm fuzzy clarity" that you get when you know that you have completely satisfied the project requirements).

All the projects had accompanying walkthrough videos that were remarkably brief despite covering all the requirements. Unlike some classes, the walkthroughs do not explain anything about how to do the projects - these walkthroughs are about getting you situated so that you know what is expected of you and where your work will take place. Some walkthroughs have small tips to help you find success, but these walkthroughs (like this class) are not there to hold your hand.

• 0. This was an introduction to LLVM and doing basic tasks with it. It took me a very long time - I think some of the earlier reviews from this semester for this course came from students that got overwhelmed by this project and the other LLVM projects and dropped the course.
• 1. This was a Fuzzing lab that used LLVM. Also difficult. Part 1 was hard because of LLVM. Part 2 was a puzzle - you're given binaries to simple programs that crash on certain kinds of input, and your task is to come up with a way of throwing randomly changing input at them in such a way that they'll crash (without knowing anything about how they work - hey, that's black box testing!). There was also a significant extra credit opportunity available (I wasn't able to take advantage of it - too busy - but it looked like it was interesting.)
• 2. This was a Dataflow lab that used LLVM. What is dataflow? There's a long lecture on it...but it's basically a kind of static analysis that works through each line of instructions to look for things like whether a variable is ever used before it gets redefined again.
• 3. Datalog...this was one of those projects where you could easily spend 20 hours on writing 5 lines of code. Set theory helps with this one, but I lost some points on it.
• 4. Type systems. This was in Typescript - fun little project where you slowly added type annotations to a big Javascript library to reveal where the bug was. Of all the projects, this one was the least related to the lectures, IMHO.
• 5. Cooperative Bug Isolation. Fun little exercise where you took the results of lots of crash reports to help you find the line(s) of code that were causing crashes. This one also had peer review.
• 6. Delta Debugging. This was a somewhat more challenging exercise where you took a large text input that caused a crash in a program, and whittled it down to a much smaller and more useful text input that exposed the same bug.
• 7. KLEE. I didn't do well with this one, but it's a way of adding instructions to obfuscated code to help reveal its "interesting" behavior. Lots of cybersecurity analysis implications, and another demonstration of how LLVM is useful (although you aren't directly interacting with LLVM in this project).

Some of these projects had "gotchas" that made it tough to get full credit, but it was straightforward to get a reasonable score. The rubrics were also clear and fair.

It seems like they are pretty good about thoughtfully designing projects and making adjustments based on the learning objectives and class experience/feedback. Projects 0 and 1 were new this semester, and were introduced to help students get comfortable with LLVM (and it sounds like more of the projects are slowly moving to LLVM), because LLVM and C had been large pain points for students on one of the other projects. I think it worked - while I still struggled with the other LLVM projects, they weren't nearly as hard as they would have been.

A special note on LLVM...it's a powerful tool, but it's used as a compiler. That means if you aren't comfortable with compilers + not comfortable with C + not accustomed to looking up documentation on an unfamiliar API = a rough time. Why would a compiler be applicable to this class? I saw some complaints about this. To me, it makes sense - a whole host of kinds of software analysis are white-box and static. LLVM lets you do things like change all " < " comparisons to " != " comparisons in a program, or flag all the places in a program where a variable is assigned a new value. So you aren't actually doing compiler kinds of tasks (like you don't need to know how to do optimizations, for instance), but there's a reason why it's part of so many projects in the class.

Pace of the course/Workload

I found the overall pace of the course to be busier in the first half of the semester, lighter in the second half. Part of this is because the harder projects are in the first half of the semester, and some of the harder lectures (like Dataflow analysis and pointer analysis) are in the first half.

However, all the projects were released at the start of the semester, so you also have the opportunity to work ahead if you want.

I think I spent about 15-20hrs a week on this class in the first half of the semester, and 10-15 hours a week in the second half.

Exams

The exams are tough, long, but fair. I would say they rival GIOS in this way.

Each exam has a set of practice questions and solutions that are very indicative of the rigor of the actual test. The instructors provided videos explaining how to approach most of the questions, sometimes with extra conversation of ways to think about the questions or tips for solving the more easily.

The exams are open lecture (you can have the pdf transcript of the lecture, which they provide, as long as the only edits you make to it are highlights), and you are allowed a single digital cheat sheet (they give you a template).

The exams are tough because the questions require you to be able to demonstrate that you can apply algorithms and analysis to a new problem. Rote memorization will not get you through these tests with success - you have to spend time understanding the material. That also means that cramming for these exams is difficult. Unlike classes with lightweight, multiple choice questions (like CN, ML4T, and IIS), a quick cramming of all the lecture videos won't be enough to get you a passing score (most likely). The practice questions (which they make available from the very start of the semester) are very good indicators of how well you need to understand a concept in order to succeed when tested on it.

I think the exams were fair because none of the questions felt like trick questions, and I felt like the practice exams that they provided ahead of time were sufficient to demonstrate how the test would flow. It was also clear to see how each question corresponded with the core of a lesson from lecture - none of the questions felt like they were just trivia questions to me (and even if they were, you had the full transcript of the lectures to ctrl+F through to find the answer if you wanted).

The exams were kind of long, but not much longer than the practice exams (just a little). Hard to know exactly how many questions there were on each exam (some questions were just "scratch space", and other questions were multi-part). As some students have mentioned, it was tricky to scroll around from a question to the scratch space, and copy-paste is disabled. However, as explanations from Chris Poch will tell you, that's not a hindrance that the instructor/TA team are trying to put in your way - some students from my class submitted proposals to Canvas to make the scratch space a "pop out window" to make the user experience, and Chris voted for it. This seems like it's one of those annoying but "no better solution exists" scenarios. That said, I took the exams on my little surface pro 3 tablet, and for the most part I felt like either the scratch space was reasonably "close enough" to the question, and/or the question had a way for you to work it out as you went along. I think practicing with these limitations on the practice exam go a long ways here.

The exam windows were three hours long. I think that's plenty of time - I had no issues with the time limit, and I didn't hear of anyone running out of time.

The final exam was not cumulative.

A logistical challenge - based on some unfortunate past experiences the instructor team has had, there are a LOT of instructions on the logistics for the exam. They have an exacting description + video of what they want to see with the room scan. There's a lot of instructions on how to do the cheat sheet + lecture transcripts. There's a policies acknowledgement quiz that's important. Make sure you leave yourself enough time to do a final check on whether you're doing all the logistics correctly.

TAs and Instructors

There are two instructors and a squadron of TAs. It sounds like this course isn't taught on campus these days and is completely online. This leads to a unique situation where one of the instructors and both Head TAs are OMSCS alums, and all the rest of the TAs are either OMSCS alums or OMSCS students themselves - there aren't any on-campus TAs. I think this is a benefit because on-campus TAs sometimes have a hard time understanding the challenges that come with being online only - this TA team is fully sympathetic and acts accordingly. I was also impressed by the wide array of experience that these TAs had based on their introductions from week 1. If you desire to (I didn't), they seem knowledgeable and willing enough to dialogue on where the course content intersects with the real world in their experience.

Mayur Naik is the creator and the PhD/professor backing the whole course - I didn't see much from him, but it seems like he is involved in the background, and he recorded some Youtube videos to walkthrough some of the final exam practice questions that were uploaded in March 2020, so that tells me he's very much there.

Chris Poch is the other instructor, and is much more involved in the administration of the course. He was on both Piazza and the course's Slack regularly. Since he is actually an instructor and not just a TA, he has much more authority than normal Head TAs do. That's significant because it means you can directly interact with him to request extensions, ask for a ruling on an issue, etc. - he has the authority to make policy decisions. Ultimately, that means a faster decision time, and also you can get more behind-the-scenes information on things like, "why is this taking so long to grade?" or other policy questions. He also clearly cares about communicating expectations clearly and making changes or policies that are beneficial to the students. (For instance, he explains that they push to get grades back rapidly because studies show students learn best when they receive prompt feedback.)

I found all the TAs to be knowledgeable, helpful, and respectful. Naturally, some were more helpful than others, but I had no complaints with any of them. However, "helpful" != "holding a student's hand or giving away the answer." They are more likely to help you reach understanding via the Socratic method (in other words, by asking you questions back), than they are to drop hints.

The TAs and Chris held a consistent schedule of office hours via the CS 6340 Slack workspace in a dedicated channel. I only used the OH once, but I saw them actively helping students throughout the semester. There was generally at least one OH session per day, with some days having more than one session, and there were generally extra OH sessions on the weekends. The OHs were pretty much for helping students with projects, and like 90% of the time they involved one-on-one DMs with students.

This is not a team that will get "buddy buddy" with you (like IIS) - the team is very professional. However, that doesn't mean that they aren't encouraging or helpful.

Piazza and Slack

This course uses both Piazza and Slack. For Piazza, it got extra messy this semester as they instituted participation credit (and extra credit) for the first time. It seems like they applied the credit fairly, but I won't say much about it because I'm really sure they're going to be tweaking how it's done. They set up Piazza to have folders and instructor posts in the usual way - I didn't have too many problems navigating it. However, because nearly all the projects are released at the very start of the semester, you can have some people working on project 1 and some working on project 7 (we had a handful of students who finished all the projects before the midterm). I found that just passively marking posts to future projects as "favorites" to find them when I needed them later was useful.

The Slack workspace is not as casually active as others (GIOS and IIS workspaces are much more active), but there was reasonable student interaction. The office hours channel was busy consistently. I found it useful to just passivly check Slack here or there as students normally would chatter there if another student had uploaded a test suite that I didn't already know about.

Special note: COVID 19 response

Chris made special accommodations/announcements early (about as quickly as Dr. Joyner did in his classes). They had a system in place quickly to grant students up to one week extensions on any project, and for students to request incomplete grades if needed. He also provided a lot of information on the ramifications for taking an incomplete, and I had a few friends that ended up doing that. It sounds like Chris took good care of anyone who had extenuating circumstances.

They also pushed and opened the final exam almost a month early so that students could have a longer window. (for perspective, the midterm was only open for about 3-4 days) Chris was super open about how that added a lot of big questions for some policies (for instance, they had to cut the participation grade short because students that took the exam weren't allowed to participate in any conversations concerning open labs).

My overall sense of the situation was that they worked hard to be accommodating and understanding to students while still ensuring that they had reasonable requirements for students.

Summary

Overall, I thought this was a very well executed course. I can't speak for how applicable all the concepts are to real world work (no experience in it). I am glad I took the class, although I've decided that the topic is not one I'm interested in. Nevertheless, I learned a lot, and I'm confident I'll see some of the concepts again in other areas. I think it's very doable to earn a satisfactory grade, and that this isn't a class where one mistake costs you a letter grade (I mean, unless you do something like completely disregarding the room scan on an exam). This class is a good class to take solo or paired with a lightweight second class (I paired it with ML4T). I'm sure that there's room for improvement in how this course is delivered, but I had a positive experience. I think most people who had negative experiences had different expectations for what this class is. Just make sure that you have appropriate expectations walking into it, and I think this class is beneficial to any OMSCS student considering taking it.

You can learn some really interesting techniques during this course.

This is not a well designed class. Going in, I thought I would learn a lot about how to create and run analysis to debug concurrency errors, detect buffer overruns, different types of analysis and testing, and generally be a better software developer. However, this class is much more academic and theory driven, focused on compiler optimization. I'm 6 classes in, and so far this is the hardest most frustrating class that I've taken. While the course is run by excellent TA's, the lectures are cryptic, the labs are unapproachable, and the entire course badly needs to be redesigned

The lectures are difficult to make sense of. While it doesn't help that the professor doesn't have the greatest speaking voice, it's far from the greatest problem with the lectures. The professor goes quickly into detail without clearly explaining background information, or giving any reminders about context. Dr. Joyner's HCI lectures, by comparison, are excellent at explaining, re-explaining, and hitting important points home before and after giving detail. But this class's lectures are all written like an academic paper that's read aloud. I've had much better luck getting information about topics on YouTube, Piazza, and from other universities. If the lectures were rerecorded with much more background information and context, broken up into fewer pieces, and integrated with the excellent explanations the students provide, it would make the lectures easier to understand.

The labs themselves have a very steep learning curve. Each one begins in an unfamiliar domain such as LLVM analysis or Datalog. Understanding how to approach the labs, what you actually need to do, and how to start making progress is 90% of the difficulty here. The TA's, however, have done an excellent job of creating introductory YouTube videos to guide us through the background info. However, it's still not enough. Instead of trying to implement analysis algorithms we just learned, it would be much more helpful to add an additional labs to learn the basics of the domain we're learning about. This semester, the TA's added an extra lab as a tutorial to LLVM. Which is a great idea and definitely helps. But there should be one to introduce students to other domains like the often counter intuitive Datalog. I've gotten much clearer explanations by students on Piazza and TA's in the intro videos than anything in the lab instructions or in the lectures.

The grading for this course is very fair, but the tests are hard. All grading is done within a few days, which is much better than any of the other courses I've taken. The TA's also set up a private slack workspace just for the class. They do not have a weekly office hours recording and only respond to questions privately via Slack, which I definitely miss from other classes. There is a large focus on anti-cheating in this class, since the lab assignments are based on ones from other universities. The midterm and final require a proctortrack scan with a mirror which is honestly not a big deal. But the test itself is very difficult to do without scratch paper (the blank text fields they provide don't help, since you really need to draw diagrams).

If you're interested in compilers and how static analysis is implemented (but not used), take this course. If not, then skip it. It's only the work of excellent TA's that save this course from complete disaster.

I have to admit I made a mistake enrolling this course. Was trying to meet degree requirement and intrigued by the "test" word in the course title. Thought I could learn something about software test but end up finding they are doing compiler analysis. For guys from non-CS bg, it is too low level and like others have mentioned ppl spend more time figuring out how to make the C++ code compile than learning actual stuff. I think 99% CS people would not need this level of knowledge in their daily work.

Now debating whether to drop the course or keep on torturing myself to earn the credit so I don't waste a semester.

The course have some strict/weird rules like mirror in protortrack, participation, vague guideline on labs. They are more like communist who want to control how you behave in the course.

In short, this a compiler based analysis course. Don't waste your time if you are not interested in the topic.

This is my first semester and I was very excited about this class and now I am about to withdraw. I will try doing lab2 and if I am unsuccessful I will be done with it. Lab2 seems to be a lab from a compilers class WTH, knowing that you have students coming from different backgrounds not CS only background, they should try to do better. I am pretty sure this class is a breeze for CS majors... I am very sad right now. I agree you should know C++ for the first labs. I spent more time figuring out when to pass a reference, a pointer or an object, seems that the only way to teach Software Analysis is using C/C++. I hope the other classes in the program are not like this one.

This is my fourth course in the program and I will be dropping it. Please do not take this as your first course. You will be utterly disappointed in the whole program. The information in the course is pure academic and not useful at all for working professional. I made a mistake when I enrolled in this course. The labs are very vague and very hard to understand and this is where most of your time in this course will be spent - trying to understand what they want from you in the lab. If you decide to take this class - make sure that you are well versed in C/C++ but again - most of your time will be spend going back and forth between puzzling lab instructions and piazza. Good luck to you.

Extremely vague instructions on labs and extremely vague answers from any of the instructors. They are more scared of people cheating than trying to actually teach anything. Almost every post mentions "I'm not sure if I'm allowed to post this..." The ProctorTrack webcam is already a little over the top, but you're also required to buy a mirror and scan your room.

not that useful...

This course was incredibly well-run by the instructors and TAs. They made it a point to check Piazza daily and respond to any unanswered questions. Office hours are done nearly every night via Slack, a format that I felt worked well. Sometimes you just want a one-on-one for 10 minutes or so to make sure you're approaching the task correctly. There are only 11 Udacity lessons, and each of the 8 labs aligns exactly with one of those lessons, in order. The labs, to varying degrees, test your understanding of the lesson (e.g. by implementing an algorithm) and mostly focus on writing programs that apply lesson techniques to find errors in code. The 2 exams were multiple choice and fill-in-the-blank with a handful of free response.

The class focuses mostly on dynamic and static code analysis techniques, not about manually designing test cases or using testing frameworks. For this reason it's less applicable to most people, including myself, but I found the concepts interesting and had no trouble staying engaged. A couple of the lessons were so interesting to me that I had to show my friends at work, while other lessons weren't as cool. I see other reviews complain about the instructor's accent - I had no problems at all understanding, but the closed captions weren't accurate.

It's entirely possible to get by in this class only putting a day or two per week towards it. I'd recommend watching the lecture, taking notes to understand the material, followed by doing the lab - then reviewing the Udacity quizzes before exams. If you get caught up on some detail in the lesson you can ask for clarification on Piazza, but you usually only need a high level understanding to finish the lab. The midterm and final review sessions were very useful too. The questions on the exams largely aligned with the review questions.

The entire course can be front-loaded, so it's very flexible with any schedule. Each lab is done on the course VM and submitted via Canvas. All labs are available at the beginning of the semester, include a detailed PDF explaining your goal and how to start (with exact terminal commands for compiling, etc), and have an associated video where a TA walks through the setup and explains some of the code. They often tell you exactly which files you should be changing and which files to leave alone.

Pros:

• Excellent maintenance (Piazza, Slack, response times, feedback, guidance)
• Can use lecture slides during exam and one page of your own notes
• Exam reviews and generous time limit make them far less threatening
• Labs are directly related to lesson material
• Flexible, can work ahead

Cons:

• ProctorTrack mirror scan - It's invasive, but I had no problems
• Applicability of material
• Jumping between languages for each lab
• Cutoffs were a bit high. ~90% for an A and ~80% for a B.
• Labs range in quality and some feel just like a game of "find the bug"

This was not exactly a course on testing, but the ideas and the tools that were created around testing.

It helped me understand the ways a language helps you check for errors, how limited some types of IDE checks can be, and even help me a little in my work life in day to day programing.

I really enjoyed the course. Learned about different program testing methods. I’m not a fan of Datalog but I see it’s value in testing. The Datalog project was the hardest for me. But most of the other projects where either easy or medium difficulty. The exams are thorough, so make sure you study and under how the projects work because there are questions on the exam that will walk through a senario similar to parts of the projects/lectures.

This class is a fairly easy class if you have certain experience of programming. It deals with various programming languages (C++, Java, JavaScript, etc) but they are all superficial and light. There are 8 labs and 4 labs are mandatory, and you need to take 3 more labs among other 4 (If you take all, then one that has the lowest point is ignored).

Tests are fair. During an exam you can see a lecture slide and bring one page pdf in which you can write anything in advance. Also sample problems are provided. You can consider that actual problems are a bit more advanced than the sample ones though. I didn't like the Proctortrack policy, you need to scan your room 360° however ProctorTrack is not so friendly to perform this...anyway I recommend you to buy some cheap webcam and use it. It is easier than using a mirror.

Content-wise, I don't like it much. I expected the class to focus more on analysis side but it focuses mostly on testing side. I was surprised that "type system" in this class was taught just by using a query annotation in TypeScript in a lab ! I expected to learn compiler / interpreter level of type distinguishability and memory layout etc, more system programming oriented stuffs.

Also I feel the lecture takes too much time to explaining simple ideas, kind of wasting time to watch. I feel audiences are non-tech people while watching lecture videos. Labs have a similar issue, some of them require you to find a bug in their code but the code-base is messy and tough to see what it wants to achieve. So you need to spend time not in finding a bug but in analyzing the spaghetti code.

If you want to get easy A this class is one candidate. Or if you are not currently in the industry and taking this program to get in, there might be something to learn. Otherwise, I don't strongly recommend this class.

Fascinating course. Labs are challenging but actually very relevant to the content, and the tests are also challenging but fair and balanced. This course is incredibly well run. The TAs operate like a machine. Grades come back lightning-fast, and contact with the teaching staff is easy and readily available. This is a great first course in OMSCS.

This is a surprisingly decent class. The instructor is great and presents well. Everything you might want to know about the course is available at rightingcode.org.

The assignments are all pretty good. They are 90% computer-graded, so make sure you follow the instructions precisely. I don't know how useful I will actually find the techniques we learned here will be, but even so, I think this was good content to be exposed to. Everything is well-designed and smoothly run. If you are looking for a class to pair with something harder, this is a good candidate.

Very responsive instructors. Interesting projects. Quickest grading I've seen yet.

The lectures are OK. They are a bit dry and hard to follow at times (especially when the sentences are looooooong). The projects are released very early so you could in theory front-load the whole course.

This course content is pretty awful and useless unless you're doing QA type of work. What led me to drop the course was a combination of confusing lab instructions, horrible virtual environment setup, poor lecture videos (the professor has a very thick Indian accent which makes it very hard to follow what he is saying) I had to resort reading the powerpoint notes instead of the videos. I would not recommend this class to anyone who does not have prior C++ and more importantly compiler level experience. Working full time and taking classes at the same time is stressful enough, I would rather not add the stress of getting failed by random gotchas. Most of my time was spent learning the tools they introduce and getting the laggy virtual environment to work. Another tip, if you end up taking this, do not combine with another course! I made the mistake of signing up for this course because it was one of the few open courses during my registration period and I regret that decision.

Was not bad by itself. Tests are "open note" with specific guidelines. you have 3 hours to take the tests, You could potentially take the full time.

The course material is very simple and I liked the video course content. To make for simple content, the lab instructions were less clear and somewhat omitted instructions. Apparently, I felt like the goal was "gotchas" rather than programming skill that was presented.

So I dropped from the course so that I can concentrate on other courses that emphasized learning rather than dealing with snags.

Course content is pretty mediocre, although what led me to drop the course was a combination of having the most aggressive proctoring rules in OMSCS and confusing lab instructions. Working full time and taking classes at the same time is stressful enough, I would rather not add the stress of getting failed by random gotchas

Great class! The topics covered quite a breadth of material that dangles between being theory and application. If you want to dive deep into software, then this is the class for you. BTW, this class is not about unit testing. The class is organized and runs pretty smoothly. The TA's are responsive, and they actually release all the labs within the first few weeks so you can always work ahead, which is something I wish most classes did. The labs were not too bad. Each lab is very different, so you are bound to have some favorite and least favorite topics. I felt that the labs did help reinforce the learning. Most of what they want is outlined in detailed lab instructions, but you really gotta make sure you read them. I did poorly on one lab because I didn't follow instructions, but fortunately, they drop the lowest grade.

There is a midterm and final, but the final is not cumulative. They have practice exams, which are quite similar to the actual exam, so make sure you look over them. We had the professor give us an exam review session, which was incredibly educational and helpful.

Very interesting content. It's not about unit tests; it's about writing programs to analyze programs.

The labs are all very straightforward, but cover a broad range of content. Difficulty of this class depends heavily on how broad your own experience is. You'll be using C++, Java, Javascript, Datalog (strong SQL understanding helps here), lots of command line utilities, and probably more. If you already have experience with most of those, you can cruise through most of the labs early on. Otherwise, plan time to get up to speed.

Big thanks to the best and most resposive TAs of any course I've taken in OMSCS so far!

The workload is rather heavy for a summer course: 7 labs, meaning almost every week, there's a new assignment. They are all challenging, but you get the benefit of samples provided to check your work. You can also check your work against data sets provided by other students.

The reason I gave this a neutral rating is because of the exams. For a person who typically gets A's by studying hard, be prepared to broad-sided by a big fat C. The way some of the questions are written seemed arbitrary. You might have to type in an answer defending a "yes" or "no" answer to a question that could be answered by either yes or no. So depending on the aptitude of the TA grading your course -- who may have a different opinion on the answer -- you might take a hard hit if the TA disagrees with you. This is a case where over-studying can actually hurt you because you start seeing the nuances and blurred lines of best practices, whereas the instructor/TAs are expecting you to think in black-or-white, and no amount of arguing will get them to change your grade, even if it's clear you know the material probably even better than they (the TAs) do. The fact that the average on the exams was a C across the class speaks volumes as to the fact the tests are either designed to trick or questions are framed half-hazardly not explaining enough to give you context to answer correctly. Fortunately, if you get all 100%s on the labs, you can survive Cs on the exams and still get an A.

This is a pretty great class and likely will spoil you - The instructor actually responds to piazza. The TAs hold office hours 3x per week and answer any questions. Oh, and grades were in within 3 days in all assignments except one that did a peer review component (Lab 6).

Other than that, the information is pretty great. I took information from this course and handed it over to a chief architect to review (specifically, papers about the processes MS/Google use to implement new testing strategies). I learned techniques that I'm wondering how I can apply them or teach them to my peers. I even learned how some commercial products that we use work.

This is not about unit-testing.

Some labs were challenging and they certainly take time to get to your "a-ha" moment, but they're all worth it.

Delta Debugging is my favorite topic of the course, by far.

Solid survey type course on static and dynamic analysis tools and techniques.

To the extent that there's difficulty, it lies in rapidly moving between radically different programming paradigms for projects, but if that sounds fun to you, you'll do well.

Strong tooling provided by prof/instructors/TAs for projects. At least in my cohort, this was supplemented by a broad range of students contributing secondary tools/checkers.

I entered with an interest in the subfield and left with a bunch of ideas about techniques to explore/implement for hobbyist work and a better grounding to evaluate existing tools for usage in professional work.

The class material is complete garbage, there is absolutely nothing useful to learn. The only reason I would suggest this class is for an easy A, or to double up a semester to get the program done faster.

The work load for this class is very easy. I took it over summer, there is a lab each week, a midterm and a final. The labs are pretty straight forward, and they drop the lowest grade. The midterm was somewhat challenging, but you are provided preparation questions that prepare you well enough.

Great info on static and dynamic analysis of software, plus some fun QA techniques.

Class is well run and some of the projects are interesting, but the subject matter is so dry. Although I found the information to be useful, I struggled to stay interested. I don't have a formal CS background so I thought this would help fill some gaps in my understanding of software. I think it did really help a lot, and I may not have focused on the topic if not for this class. The labs do a good job of introducing some great commercial and academic static and dynamic analysis software. TAs were knowledgeable and helpful. Piazza was useful which may have been a result of (1) small (~250 ppl) class size (2) responsive TAs and regular updates. Homeworks were graded quickly. I enjoyed the tests even though they were challenging because they deepened my understanding of the topics presented. I paired this with a tougher course and was able to complete all assignments and one of the two extra credit assignments.

This course was exceptionally well-organized, with the most responsive TAs I've seen so far in this entire program.

The lab instructions were very clear and the evaluations were fair. The contents of the lab were generally interesting. They will take you through low-level debugging concepts involving pointers in C++, to analyzing type systems in Java. Generally, you will not need to write that many lines of code for each lab, but I found that completing them really required a solid footing in the material taught in the lectures.

What I liked most about this class was that it dealt with mostly analysis at the "meta"-level: writing code to analyze code. The class is as much about how programming languages work as it is about debugging and testing code. This is a very rich area, as I have learned, and made me appreciate much more what's involved in how we make sense of code.

Great first class for the program. The theory was interesting and presented clearly, and the labs backed up most of the modules with practical application. Exams were challenging but fair. TAs and professor were available on Piazza and for office hours. I think this would be a good summer course too.

Here's how the labs broke down for this semester:

1. Fuzzing (pretty easy)
2. Automated test generation (pretty easy)
3. Dataflow analysis in C++ (tougher)
4. Dataflow analysis in Datalog (difficult conceptually but not too much actual work)
5. Statistical debugging (tricky but cool)
6. Delta debugging (pretty easy)
7. Symbolic execution (difficult)

Learned a few things that would be useful for work, like using datalog to analyze java code and a few analysis and testing techniques. THIS IS NOT A COURSE ABOUT TESTING LIKE WE DO AT WORK (unit/e2e/smoke/etc.)

Disliked having to parse recursive algorithms manually during test via proctortrack , with over 20 droddown list to fill without the use of any paper. That was just ridiculous

I thought that the course we well organized, I got a lot out of doing the homework/projects. They can be a bit difficult to approach, but thought that the lessons and the videos that go with each project were helpful to get started (definitely watch the office hour videos that go with each assignment). However, the aspect of the course that I did not like were the exams. I thought that there was a big disconnect between the material from the lectures and the exams. I felt like the exams were something out of left field. And the only way to prepare for it is by taking a mock exam. Most of the questions are similar, and so it's a must to go over the exam, study the type of question being asked and understand the answers. But I felt that there were not enough examples to help reinforce the concepts. There were one to a few questions per concept, but it didn't help much in the end. I wouldn't recommend this class for first timers.

The algorithm and techniques thought in this class is used in industry if you work in compilers, javascript engines or security teams. So its useful if you target one of those.

TAs help would have been better if they are available over slack most of the time rather a office hours. Office hours close to assignment dates were kept with long gaps. They provided sample questions and prof. goes over them for both the exams.

The assignments are few lines of code but too complex to derive due to syntax and understanding on the algorithm taught here. I put very little effort compared to my previous 2 classes and did good. C & C++ & Java used but never needed to write a lot of code but few difficult lines.

Background: Mechanical engineering undergrad, not too much depth in different programming languages, but work as software engineer.

I have taken 2 classes before this. Paired this class with CS6300 SDP. I would suggest pairing this with a medium difficulty class (maybe something a little harder than SDP, but nonetheless it is a good class to pair with). I didn't like this class as much because the material was pretty dry. The class assignments have many different programming languages (Java, C++, C, Datalog, etc.) You won't need to be super good at any one language, but just know how to code and figure it out. What's nice is they have a VM set up for you so you don't need to worry about all that. Lab videos and TA's are super helpful. I encourage having a classmate to talk to and discuss topics because it just helps understanding, and makes the class more enjoyable.

Exams aren't TOO hard if you pay attention to the lectures and review courses. Doing the practice exams will prepare you well. Exams are curved as well. Got 90+ on all of the assignments which shouldn't be too hard if you did them correctly. 92 on midterm (curved and somehow lucked out) which was above average, and a 69 on final which was slightly below average. That gave me an A.

What I didn't like about this class was that it was mainly about figuring things out... they will introduce new things frameworks and API's that you've never seen before but you need to know how to read documentation and learn quick. I guess this is the skill a Software Engineer needs to have because wherever you work its going to be different so the skill of knowing how to figure new things out quickly is good. It was a lot more "figuring out/analysis" than coding. Sometimes the assignment is just a few lines of code.... literally like 5 lines of code, but knowing how to write those few lines will take a long time to figure out; that's what makes it a little frustrating.

I am sure that the things you learn in this class can be applicable and important, but IMO it is also the boring side of software.

Every week is basically a completely new material, and a lot of it is good (ie, practically unknown, but very interesting and potentially useful). Labs can be mundane to sadistic.

Quality of lectures leaves a lot to be desired - they are basically handwaving and expect a deep theoretical understanding in the exams. I frequently had to refer to lecture notes from other univs CS courses to get an understanding.

eg the exams (quite tough IMO) go very deep into soundness and completeness, bu the lectures do not explain them in any depth.

I hated this course with my whole heart. But that might be because I am not that into Testing/Analysis. I registered in this course as I was not able to get into any other course due to huge WL but regretted the decision whole semester. I did my labs really well but to get A you need to perform really well in exams (which contribute to 40% of the grade). So getting an A is really hard. Each lab is a new one, so may have to spend lot of time understanding the lab first and then work on it. Few labs are very easy if you are in right path otherwise even the easiest ones may become real hard. I hated the fact that even after scoring labs well >98% in all of them I ended up narrowly missing A because exams were really hard. First of all they are pretty long (3 hrs), I hate doing that at this point of my life. I mean if we are already investing so much time in labs, they must be rewarded well. The no scratch paper policy for exams was irritating. Also I didn't like the fact that all the labs were given equal weightage. I mean there are couple of really hard labs and couple of very easy ones. It's not fair to give them equal weightage. Overall I felt like I was in high school again where practical knowledge (Labs) is less important than theoretical knowledge (Exams). This is the only course I got a B, so I regret taking this course. Take this only if you are really into Analysis & Testing. You will need C and C++ knowledge to succeed this course. On a plus side, the course was very well organized. The past Head TA now promoted Prof Chris is really good and always replies to student queries really fast. Although there are very few Piazza posts compared to my other courses since most students engage in private discussion with TA's on Slack. But Both the TA's, Professors and many students seems to be really involved in the course.

A lot of the work is doable but a lot of it is not. This course used C, Java, Prolog, bytecode in a Linux env. work for labs is auto-graded so if you are 99% finished and turn your stuff in the grader will give you 0% for your grade on that assignment. If you do not know all the tech used very well you will spend tons of time learning it. This is not a good first class to take. No Extra credit is given.

This class was a ton fun, really interesting, and very well run. It is not about unit testing. Seriously, y'all need to stop with the "I dunno why SDP covers so much testing since there's already a class just for testing." Different testing. Also, the old reviews of SAT complaining about broken or outdated this or that are no longer applicable. Woohoo!

I took this with AI and got a solid A in both. So, you can as well. I previously completed AI4R, CV, ML4T, IIS, and RL. I would probably put this as the second easiest class I've taken, behind IIS.

This semester we had 7 projects and 2 proctored exams. The projects are likely to change, but we had:

• Fuzzing
• Randoop
• Dataflow analysis
• Datalog
• Cooperative bug isolation
• Delta debugging
• KLEE

The midterm and final were both proctored through ProctorTrack and both allowed one page of your own notes in addition to the instructor-provided notes for each lesson. Make no mistake, you absolutely should still study. I found the midterm to be relatively difficult and I thought the final (non-cumulative) was a lot easier. Not because of format changes or anything like that. I think I just found the material in the second half of the course easier and more enjoyable.

Check out rightingcode.org if you want more info about this class. All lectures are available there and are easier to access than on Udacity.

There was no required textbook and there were no required readings in general, but there were a lot of recommended supplemental items.

Instructional staff was very supportive, including the professor, which was unexpected since he's technically a full-timer at a different university now, if I remember correctly.

Overall this class was a lot of fun and something very different for me. I'm glad I took it and I strongly recommend it. It's good to pair if you have the right background.

Speaking of background, check out the prerequisites at the link I gave above. Take them seriously. If you have a CS undergrad degree you should be fine. If not, it's not too difficult to get the additional background you need, and it's something you should do anyway if you want to survive a grad-level CS program.

Course videos/resources did not align with assignments/tests, which made the course challenging. If you are familiar with C and scripting the course may be a good one for you.

• Extremely well organised.
• The course focussed more on the analysis side than on the testing side.
• Involved many programming languages: C, C++, Java, Datalog. It might be challenging if you don't know most of them.
• All the assignments required a lot of thinking and analysis, but once finding out the solution, they were easy to complete (few lines of code).
• The topic itself is very interesting. Even if you don't want to work specifically in analysis, it will help you to be a better programmer.
• Instructor and TAs provided excellent support.

I started out not liking this course but it started to grow on me more into the semester. It is NOT just writing unit tests, it is more tied to the analysis of code in general and touches on some interesting topics around testing. I enjoyed doing the labs, they can be challenging(the Datalog one was particularly cool IMO). The exams are hard, and worth 19% each - but fair. If you make a good study guide, understand the concepts, and watch the TA video before you should do fine. They take the best 6 of 7 labs which worked well for me because I totally forgot about an early lab. Ended up with an A and I wouldn't say it was the most time intensive course.

This was my third class and I didn't like it as much the other ones. The labs are not a lot of work but it takes a lot of time figuring out. There are a couple of labs that take a lot of time just trying to figure out. Each lab is independent so you might do good in some and not so good in other. I ended up not doing one just because it was a lot of work (6 out of 7 count). Start early and try to do the labs that last a week as they are the easier ones. TAs during office hours were not really helpful, I found more help from fellow students. I did find the videos for each lab helpful. I failed my final but the midterm was way easier so ended up with a B. Maybe I just didn't have enough time to study. Definitely study the practice tests and ask questions during office hours for both tests. I don't find anything really useful out of this class besides KLEE ...if you use C. Also, this class is very intense for C if you haven't used it in a while..I didn't so it took me a while to ramp up.

I highly recommend this class. It is well run and difficult but for all the right reasons.

Class structure is 7 labs 6 of which are kept (10% each), a midterm and final (19% each). If you do not know C++ and Java well, you will have to get up to speed quickly. This was my situation and I got by with an A, but definitely had to put in extra effort. It is heavier on the analysis than the testing. You don't cover unit testing or anything basic. The lectures rely a good bit on formal logic and outlining theory, but the labs have a direct implementation of each. The midterm/final were tough but fair questions.

You will walk away from the course with a number of different automated analysis techniques well explored. The course staff really worked hard to make sure students had everything they need to do the work without completely holding their hand.

Uses a variety of programming languages but you only need to know the basics of Java and C. Interesting material on fuzzers but complex topics. Tests are really difficult but the projects do help you learn, although they don't always relate to the lectures. The lab on datalog is extremely difficult, so spend some extra time there.

Overall, an excellent course. I had taken Compilers the semester preceding this class, and found this had a lot of overlap, but was a lot less work. (It could be used to prepare for some sections of Compilers, if desired.) This is a good class to pair with another light-workload class, or with a heavy-workload class; I only had to spend 0-4 hours/week on this, and was often ahead in this class.

All lectures and labs/projects became available at the start of the class. We were free to move through the material at our own pace (and people did move ahead faster than the syllabus). The lectures were helpful and informative, and covered a wide range of topics in software analysis. The projects were very related to the lectures and were very manageable - the staff put in lots of effort to ensure that the administrivia go smoothly and we can focus on the concept we're trying to learn. I finished some of the labs in 2-3 hours, while the longest one took ~10 hours.

I didn't do as well as I'd have liked on the midterm and final, but the projects were enough to offset that; and we were allowed to drop our lowest project grade.

Personally, I would have liked to cover type systems in more depth, and devote a project to them. I'm told that such a project is being developed, so maybe you'll get to do it instead. :)

I went in with low expectations for this course topic but I was totally wrong. Wonderfully integrated course with the lectures and the projects complementing each other fantastically. The TA and Prof are very responsive and the course material was fresh introducing a wide range of new tools that can be utilized in current day work environment.

This is a much different and better course than the previous semesters may indicate. The projects are interesting and engaging with some much harder than others, and you learn useful tools (depending on the languages you work with--most tools apply to C, C++ or Java. The new project intro videos help you to quickly get acclimated for each project. Chris Pouch is co-professor and he is excellent. I found him tremendously helpful in office hours (Slack), on Piazza and in the Q&A sessions he hosted. I got stuck on several projects and his hints were instrumental in me doing well on (and completing) the projects. As long as Chris is part of this course, I recommend taking it, but you really need to be familiar with C/C++--at a minimum, I recommend taking IOS (CS6200) as a prerequisite (also an excellent course with an excellent professor); in addition, go through the Essential C/C++ course on Lynda.com. The TAs, for the most part, were helpful as well, although they weren't always sure how much they could help. In the interest of full disclosure, I got an A (but barely).

Note that this course should really be called Software Analysis; there was only one project that concentrated on testing--this is -NOT- a course about writing unit tests.

Unfortunately, this course continues what I consider a disturbing trend in OMSCS: students are hyper-cautious/paranoid about helping each other due to potential honor code violations. Some courses (IOS when I took the first run of it in Spring 2015) are a little looser when applying the honor code, but most courses are so strict with it that it makes public Piazza and Slack help request/question posts pretty much pointless. The result is that the only replies worth anything are from the professor and/or TAs.

I completed SA&T the summer of 2018 as my fourth course in OMSCS and will say it was the most well-directed, organized, and supported course I have taken in the program so far. The instructor and TAs had great attitudes about the class and material and were active and helpful on Piazza and Slack. The lectures were understandable, interesting, and directly related to the labs and exams. The labs were prepared very well with an almost flawless VM. Preparing each lab's file structure on the VM and collecting the deliverables was not painful but not seamless either. The exams were fair but challenging.

The course material has a heavy emphasis on Software Analysis vice Test. While the majority of the material is still interesting and was mostly in line with my expectations, I was hoping for more material on best practices of testing in the industry. The third and fourth lab (Dataflow and Datalog) were quite challenging, and full understanding is needed for a good grade on the exams as well. Dataflow and Datalog are worthwhile topics, but I would have benefited from more complex examples or references of these topics.

Overall I would highly recommend taking this course. Being familiar but not fully proficient with C, C++, Java and bash scripting was enough background that I felt comfortable with the course prerequisite knowledge. I ended up with a B because I put the hours in for the labs and did just well enough on the exams. I would have rather taken it in Spring or Fall to get more time to digest the material and have had a chance at an A.

I've been working as a software developer for 25 years, and I was expecting the class to be about the software testing process, and it was. However, to my pleasant surprise, it was a course on newer technologies (what? things have changed since I got my degree 25 years ago?) and how to leverage those technologies to effectively test and track down issues in software. The lectures (on Udacity) were well done for the most part. The quizzes (in Udacity to test your knowledge) could have been clearer, but I took them a few times to get a better understanding. No textbook. All the materials are through Udacity, though there are PDF and Powerpoint equivalents for you to study offline. There were 7 projects and for each one, a video intro is provided as well as a PDF providing detailed instructions on how to implement the project. The instructors provide a VirtualBox VM to use for all of the projects and I found it well put-together. Instructors (professor + TAs) were readily available and supportive. Knowledge of C or C++ comes in handy for a couple of the projects (pointer arithmetic). Projects require minimal coding in either C, Java, or scripting, along with one project using datalog. Lots of support through Piazza forum + slack channel. Office Hours were done using Google hangouts and if you missed it, you could play it back.

When it came to the midterm and final, while a practice midterm and practice final were available, I felt they weren't representative to the exam itself. The midterm practice exam was actually harder than the actual exam (that's ideal, I'm overprepared), while the final preparation left me feeling overconfident since the final was much harder than the practice final exam.

Overall, I enjoyed the course and have many new skills that I'm looking forward to putting to use on future development projects.

This course was not what I was originally expecting...but turned out to be very interesting and informative nonetheless. I was expected an emphasis on test techniques, but the course was more focused on the "analysis" than the "test". However, the analysis has some important foundations that are used for testing (as well as compilers and IDEs), so all in all a good course.

Most of the projects were fairly straight-forward, though some difficult to tell "when you're done". Only one was difficult, but that largely due to an unfamiliar language (Datalog). A few of them had reports, but were fairly straight-forward.

The TAs were great, especially Chris. The projects often came with videos to get you started. The VM environment for the project(s) were stable and only had 1 minor missing component on one of the later projects. Thankfully, only 1 VM required for all the projects.

My only negative on the course were the exams. While sample questions/solutions were provided, the exams didn't reflect them very closely. The exams did expect deep understanding of the topics. In some cases, I felt the exams were more complex than the lectures or the labs.

All in all, glad I took the course...learned a lot I likely wouldn't have picked up anywhere else, and have already been able to use some of the concepts in "the real world".

The worst class I have taken. Do not think I will really use anything I "learned" in this class.

I really enjoyed this course and was exposed to a lot of new analysis and testing concepts.

Overall it was an interesting class but there was usually a disconnect between lesson videos and the assignments. Some of the assignments were extremely frustrating with abstract concepts I have never seen outside of the class and likely never will. TAs were overall helpful when they could be but at times were frustratingly vague. The final is non-cumulative and only covers the back half of the course but I this given the format of the class, more tests covering smaller groupings of the material would have been a better fit.

Projects were great and fun. Some were harder than others. The harder ones were worth more points. I completed them all. I think I got close to 100% on all the projects. You need to know C and C++ pretty well. Pointers to pointers, dereferencing, etc. Or you will struggle.

Professor and TAs were some of the best I've had. The professor explained some concepts in the mid term office hours that absolutely helped me to better understand. He was great and patient and very precise and literal. I liked him a lot.

The exams were much tougher than I expected. I got a B on the midterm (just barely) and very low grade on the final. Really understanding type systems kept me from totally flunking the final. It was much harder than the midterm.

Final overall grade is curved. I ended-up with a solid B. I could have gotten an A if I had spent more time preparing for the final, but I was sort of cruising at this point as I knew no matter what I would probably get a B since I has aced all the projects and got a B on the midterm.

Good luck if you take it. Study more for the exams than you think you should ;)

Software testing and analytical skills is a must for security application developers. It should help software testers to reduce bug count before deployment and manage bugs when software is in production. Back to the course. Basically it covers 3 types of software testing & analysis methodologies: static, dynamic and a combination of the two. The coding assignments overlaps reinforce the lectures. Some were very challenging; however they provide output goals for each. Personally I didn't have a problem doing the coding assignments, the exams were more challenging to get an A in. The class average on the 2nd exam was approximately 68. Think critically about the lectures.

Quick background about me that might help understand the context of the review: I am a software developer working full time in a corporation (12+ yrs). Im quite comfortable coding large pieces of backend code (servers) mainly in C/C++.

With that said, here is my take on this subject: I found this course very well organized. The lectures correlated well with the projects and was quite informative on the latest trends in the software testing field. Although the projects were quite straight forward, I found the writeups associated with the projects a little tricky. Especially when it came down to grading these projects, i found some of the answers content to be subjective

The exams a bit challenging and needed some solid preparation before hand. Getting a good grade in the exams required a good understanding of the topics covered.

In summary:

1. Very well organized course and lecture videos
2. Very responsive and helpful TAs and prof(s)
3. The content was informative and the up-to-date tools used provided useful insight into incorporating them in real world scenarios
4. The exams were not multiple choice and was not easy. It required a good understanding of the topics covered.

Despite what the previous review said, you get as much out of this course as you put into it. The TAs were very responsive (usually my Piazza posts were responded to in less than an hour, and they were also almost always available on Slack for quick queries) and most of the time they were helpful without giving the lab away. Having said that, one of the labs was slightly problematic in that you either understood it or you didn't, and there was no help that could be given (other than "don't overthink it") without giving it away. However, this lab is being reworked for next semester, so that should no longer be an issue.

If you come into this class without knowing C++ (pointers and STL containers), C, Java and shell scripting to some degree, you are going to have a hard time. However, if you know these already, the class is not that hard. Some of the labs are head scratchers, but once you figure out what you need, you should have no difficulty completing them.

In summary, it's an interesting course, and, if you know how to program (and if you don't why are you doing a masters in CS?) then you should have no difficulty passing.

One other thing. Although I put down six hours per week, I actually did 11 hours some weeks and 1 hour on other weeks (watching a lecture) because the labs were due every two weeks. But that distribution has more to do with me putting off the labs until a few days before they were due rather than any issue with the course.

one of the worst course I have ever taken. Both professor and TAs were disinclined to help students with very less participation and clarification. I doubt if I will ever use topics learnt in this course in my professional career. regret taking this course after putting so much of effort and time.

Personal Background: Completed undergraduate degree in computer engineering in December 2014 with minimal CS training in Java, C, and C++ in basic concepts as part of degree. Worked as an electrical engineer in power distribution between now and then. Full-time online student currently. No formal software analysis and test training in academic or work environment. This was one of two courses that I took in my first semester.

Course Experience: Overall, the course was worthwhile from my personal background. The instructors and TAs are truly excellent in operating the course and helping the students understand the material. The assignments are very effective in teaching concepts and new tools. All horror assignments alluded to in previous semesters have been removed. Some tools were specific to academic research and not necessarily widely used in industry, which may not be beneficial for some students. Everyone had to fight through the difficulty of obscure environments and languages. I had an abnormally large and painful learning curve for most assignments due to my deficient background, hence the 25 hours a week. Five different languages used throughout assignments, but I was able to figure out the appropriate usage of each given enough time. Once you endure through the learning curve, you realize the simplicity of the coding for all but one assignment that required a specific algorithm that I hadn't implemented before. The concepts and analyses are relevant as well. The exams require application of the knowledge rather than simple memorization. For example, given this type of analysis, find the output of the given simple code rather than recite the definition and characteristics of an analysis. I gave a "liked" rating as the lessons I learned on the assignments and exams were truly valuable but often cost me 20% or more on my grade, which can be a difference in a letter grade. Then again, one should have to earn an A.

This was my first course on the OMSCS which I liked overall. My background in Java helped out a lot in the assignments but some of them were pretty tough such as working with Datalog. TAs and instructors were really helpful on Piazza and would promptly answer any questions raised. Midterm and Finals were challenging but very fair and would focus on testing the understanding of the concepts taught in class. I liked that there were a lot of assignments that tested most stuff taught in class to help in understanding and getting to use real live tools used in industry to showcase the concepts taught in class.

This was my first course in the OMSCS program.
This course provided an introduction to a variety of software analysis and testing methodologies. The tests were tough but representative of assignments and lecture materials. The assignments used a variety of programming languages (which I thoroughly enjoyed) and gave insight to the usage of the techniques from the lectures. The assignments at first pass came across as difficult but after reviewing the assignment instructions in more detail, the instructors gave plenty of hints and suggestions of how to proceed. The use of Piazza as a forum made getting information from instructors, TAs and fellow students simple and convenient. The instructors were responsive and helpful as were the TAs. Additionally, using Office Hours linked to a Piazza post with times into the recorded videos made finding answers to questions convenient and relatively pain-free. Since this was my first course in the program I don't have much basis for comparison, but I can't think of much that could have made this course better.

This was my first OMSCS course and overall I liked it. I have a background in Java and C/C++ and that helped me on the lab assignments. The hardest part of the class for me were figuring out how to start some of the lab assignments we had. A few of the more difficult labs felt like we covered a topic briefly in a lecture, and then we had to immediately work on a full lab regarding this topic. This led to me having a hard time initially figuring out how to start a few labs, and I only made it halfway through lab 5 because I had a hard time understanding Datalog. Even though I didn't do well on lab 5, I did fine on the midterm and final (which I felt were both challenging but fair, the final wasn't cumulative but was harder than the midterm) and got a B in the class.

This course was very hard, but if you want to be exposed to some bleeding-edge research on analysis and testing techniques then I highly recommend this class. Ok, bleeding-edge might be a bit of a stretch, but you will learn something new :-). Also, fair warning to those considering this as your first class. This class can take a lot of time depending on your background. Don't let the average workload fool you. My background academic experience consisted of predominately electrical engineering courses sprinkled with skeleton computer science code problems during my undergraduate program. My current job is a System Administrator and I hardly touch code. I can definitely say I this class took a lot of time due to my background. If you are like me and you don't have a decent amount of software development experience, I would recommend taking SDP before this class. You definitely don't have to take Software Development Process before this one to excel, but I believe SDP would give you the exposure of typical bugs software engineers/software developers would encounter. This exposure would give the student a deeper appreciation of techniques presented in this course. I believe anyone can take this class and do well if you work hard; however, to shave off time of some of the projects it will serve you well to have a solid grasp of C/C++ i.e pointers, pass-by-reference, STL, etc. and Java, some graph theory terminology i.e. node, edge, path, directed etc, algorithms of common data structures such as trees, linked lists, and arrays, very basic probability and statistics, and all of the common operations from Set Theory. If you decide not to brush up in these areas you can still do well, but it may be a bit of a struggle. If you have questions during the course, don't be afraid to ask the instructors, they won't bite :-). If I had done this for the first couple of projects of the course, I more than likely would have had a better outcome. What can I say? This was my first rodeo so I had to live and learn. Exams are very hard and will require a good bit of study time to do well. For my class, they provided a very watered-down version of the exams as a review and that helped a lot. This was my first of two classes in the OMS program and I did learn a lot. Hopefully, this review will be of some help to you and I wish you the best in your future endeavors with this program. Good luck!

This class is hard... BUT... it is fair and passable. The exams require quite a bit of preparation and study (I studied at least 40 hours for Midterm, 25 hours for Final, and my grades reflected it [83% midterm, 74% final]), and are timed, but there is ample time given. The assignments require multiple languages, C++ (this was one assignment, and my first time with C++, and got 100%), Java, Python, Shell-scripting, etc., and using unique tools to convey the concepts in the class. The resources are excellent, test prep, pdfs of lectures, office hours are unmatched in quality of information. The Instructors are continually making improvements, and I've personally witnessed some, (I attempted class twice). The instructors are a class act, with plenty of communication, and always making sure students are given the tools needed to be prepared week to week. While I did not ace this class, and would not want to re-take, I wouldn't change the fact that I took it. I did learn quite a bit, and will end up using the concepts in my day to day job. Just don't think you'll skate by, without putting in the effort. And If you take this next semester, I'm sure it will be improved, knowing these instructors.

Really engaging class with a ton of instructor support. Highly recommended if you are just starting the OMSCS program as it eases you into the school schedule. The course content lead to more investigation into different testing frameworks and code analysis that I am able to apply at work!

There are some interesting concepts and tools introduced in this class about testing and debugging, but there are lots of cryptic academic only notation and syntax, and definitions that seem custom to the class. The videos are very light on information and it was absolutely necessary to watch office hours and review piazza since there was a lot left out. Grading makes this class relatively simple since most of the grade is based on the projects, but the tests are vary hard. The tests seem to cover topics that are either barely covered in the videos, or combinations of topics that we are expected to figure out on our own.

Background knowledge: Do not be fooled by the supposed required background knowledge. They said we needed to be "familiar" with "a high level programming language like Java or C++). For your own health and safety, do not attempt this class without a very solid foundation in BOTH Java and C++, and experience in C and Bash scripting would definitely help to have in advance. I went in not knowing C++ and we ended up having an assignment written in that language with a special framework that I got a 30 on because I had to spend so much time just trying to figure out the language and the corresponding special framework that I barely got around to actually figuring out the concept behind the assignment. Don't make my mistakes.

Class material: There were definitely some interesting and useful concepts sprinkled in, but I work in this field and had never heard of most of the concepts/testing techniques in this class. Either my company is just really terrible at testing, or a lot of these techniques are outdated, far too specific, or not used in the actual business world, and I think it's the latter. There was some material that was very useful, but many of the lectures were incredibly boring and slow-moving. Lots of time was spent on background and not as much on the actual concepts.

Assignments: This is what killed the class for me. Assignments were either ridiculously easy (an hour or two) or impossibly difficult (30+ hours, and the majority were like this). There were 7 total and I came to dread each release of a new assignment. For me, they felt like an enormous leap from the lectures and I felt completely lost for the first third of the time spent on the tough ones. Provided resources were extremely limited and lacked detail, and the concepts nearly always couldn't be found anywhere on the web, so I had to rely on asking the other students and TAs on Piazza for everything I couldn't understand, which was frustrating. Overall the assignments tested our ability to figure out what we are supposed to do with the tiniest scraps of information as possible rather than our understanding of the concepts from class, so they felt like a huge and infuriating waste of time.

Exams: The midterm was tough but I felt it was pretty fair. We were given a practice exam for both the midterm and final and I felt the practice gave a good feel for what the real exam is like. The class average was a 65 on the midterm, but the grades were curved soon afterwards. The exams are open-lecture notes.

Professors/TAs: One of our professors was very dedicated, consistent with office hours, and was always on Piazza helping out and answering questions. The other professor who recorded the lectures showed up for one office hours, which were supposed to be weekly, and then cancelled each week after that before finally not scheduling them at all and being replaced by rotating TAs. The TAs were helpful and seemed dedicated to helping the students, and they seem like they had to carry a lot of the weight, so I commend them for that.

This was my first class in the entire program and it had me questioning whether the program was right for me. I am hoping I will have a better experience in other classes. Overall, I can't necessarily recommend the class but if you are better prepared (see section 1 of my review) you will probably suffer much less than I did.

Not a good class.

Material: The first half of the class teaches you some outdated APIs with little to no documentations, which feel like some orphaned research project. I fail to see how they can be any use once you move outside of academia. Second half teaches you tools that may actually be useful, but less than 50% good does not make up for the horrid 50%. Assignments: Extremely frustrating. Some assignments are extremely easy while the some are very difficult, not because you learn some difficult yet interesting concept, but because they are designed to be - otherwise all the assignments will take only a few hours to complete. Most assignments are auto-graded, so if you have typos in your typed answer don't expect any leniency. There's no Slack chat for the class - again if one existed the assignments would have been super easy. Some can be completed by "throwing random inputs at the wall and see what sticks", which is actually a valid testing framework, according to some lecture. Lectures: Difficult concepts are skimped, while lecturer spends a lot of time explaining obvious, common-sense concepts. Lectures are mostly disconnected with assignments. Expect maybe 10 minutes of useful materials and 30 minutes of filler contents, and unlike ESPN the filler contents can give you a coma. Examples are not adequate so the office hours are mandatory if you want to understand what's going on, which you really don't since.... Exams: Completely different from the samples and the lectures. I mean, seriously, if I didn't study Statistics in college (yes, statistics) I would have easily lost 25% on the exams. Esoteric concepts from lectures were drilled, so you have to really think outside of the box. It's excellent to get students to drop the class and apparently so, since piazza traffics pretty went quiet the weeks after the midterm. TAs and Professor: The Professor did not show up at all so we can safely disregard him. TAs work really hard answering questions and maintaining conversation but harder not revealing solutions. I feel sorry for them, since their hands are tied. The way assignments are simply prevent them from having a meaningful conversation about the concepts. Sometimes by explaining things in a non-revealing way they ended up confusing people.

Save your time and $800+ and take another class.

the class leverages out of data technology for the assignments making it very difficult and time spent on those projects. Secondly there is not enough readings for the lecture material and the lectures are very lacking. Wasn't a fan of the material in this class. Also the exams are very challenging. this class would be best suited if it updated to newer technology and provides a good book.

What a great class! I think some people are expecting a "how to unit test" type class, but it really dives deep into how software can be analyzed at both the static and dynamic levels. There's a lot of depth to this area, and I feel like this class gave me a foundation to dive into the theory and application of so many techniques.

I took this course with the thought that it would be about software testing techniques at the source code level. Although the first few lectures covered that material, the course then proceeded toward compiler-level analysis of source code. This was not what I was expecting/wanting from this course; I previewed the remaining course materials and decided that this course wasn't really going to be applicable to my interests/career. I've decided to withdraw from the course. I'd say that this course is more oriented toward OMSCS students whose background is in CS and compilers rather than a software engineering type. I thought the lecture material was fairly dry, and failed to provide much context for the concepts and algorithms being presented. I completed the first 3 assignments and felt that they were somewhat useful in reinforcing the concepts taught in lecture, but I spent most of the time trying to read the (generally sparse) documentation for the tool. This summer (2017), the instructors provided excellent midterm preparation material for the midterm exam, but the exam was still challenging (took me 2 hours to complete of the 3-hour window). I thought the grading was strangely harsh for some relatively minor errors (my opinion) in homework submissions.

I feel like the course kind of hangs you out to dry on the assignments in terms of support. The lectures cover the concepts, but the assignments are basically "Read the manual for this tool, and implement stuff. " I've taken enough well scaffolded courses to know that this is not the best way to teach people (throwing them into the deep end).

TL;DR: Overall, I was disappointed in the content of the course, since it didn't really focus much on software testing but mostly low-level analysis most suited for compilers. If you're into that type of thing, this course may be for you. I ended up withdrawing from the course for these reasons.

This was a good class. It was well organized and presented some interesting topics. There's no reading, and the assignments were typically not that difficult (there is one that requires you to learn DataLog which is tough but definitely doable). The exams were tough, but fair. The concepts presented in the lectures aren't too difficult to wrap your mind around, and I found many of them very interesting.

Some of the other reviews complain that the tools used are out of date, which is true. However, they're still interesting to use in terms of learning the concepts and thinking about how you could apply the ideas to your day to day work. Of all of the OMSCS classes I've taken, this one actually has some direct use in the industry (e. g. statistical debugging).

The class seemed well-run, with assignments being returned within a week in some cases. There were recorded office hours weekly which was helpful with some assignments.

I'm one of the survivors of this course. I'm going to try to be unbiased as possible.

1. The lectures: topics covered in this course are interesting. They don't teach you on how to use testing tools out there in the market, instead, they teach you the techniques behind those tools. Lectures are very short, with few easy examples. Most quizzes are hard to crack. You suppose to infer the answers out of the poor explanation of the concepts. In most of the lectures, the lecturer only repeats what's on the slides word by word not explaining much of the concepts. You don't have supporting materials. If you google it, it's hard to find the concepts covered in lectures. There are only few external references.
2. The assignments: what is hard about them is not what you have to accomplish, but instead trying to figure out the tools and topics under you have to perform. Topics like graph theory, pseudo machine code are necessary. You have to use deprecated tools that are uncommon in real life. It's hard to find documentation. You have to spend hours in learning those topics and tools before attempting to solve the assignments. The time slot is fair. Most of them have the expected output. Piazza is a great tool, and office hours help you a lot.
3. The exams: they are open-notes. So, it doesn't not matter how many times you review the lectures, or do the quizzes or the practice exams, it will not help you at all. Exams are very hard. It is expected that you master the topics that were barely covered in lectures and apply that knowledge to unrealistic situations or algorithms. The mean for the midterm was a 32/50. You do the math.
4. TA's are very active and very friendly, professor only showed up a couple of times in office hours. In conclusion, it's a hard class because of the way it is taught, barely covering topics and expecting from you to become an expert. Be careful if you are going to take this course.

This was my first class in OMSCS. I found it very exciting and very useful. There were 2 exams both were very tough ones. It was easy to lose marks. We could easily get about 55%. But after that it would be difficult. Assignment were tricky connects with the lessons we learn. We need to pay 100% attention to every word that was written in the lecture notes. There were 7 assignments. All were worth doing. Subject covered pretty broad spectrum of Software Testing and Analysis methods and approaches with more emphasis on compiler and Visual Studio's testing. Some of the tools I got introduced to were never heard before. Couple of those assignments were very tough. It took almost weekends to get them worked. Overall it enabled me to think in a much broader sense. I really felt the challenge of computer science subject. Overall, I felt very good about the course, although I did not fare very well.

This is one of the best courses I have taken in the OMSCS. I would put the difficulty somewhere between easy and medium, depending on your previous experiences and knowledge.

The assignments are interesting, they align well with the lectures, and more than enough time is provided. All assignments had 1-2 weeks to complete and I was able to get most done in ~ 2 days (and I have very little experience in Java).

The exams are more than fair: first, provided practice exams do well to prepare students for the actual exam; second, the exams are open-notes; finally, there is nothing tricky about the exams and the questions were entirely reasonable. Follow the lectures, take some of your own notes, keep a copy of the instructor notes handy, and run through the practice exam, and you should have no trouble with the midterm/final in this course. Study for the exams earlier rather than later. The instructors (rightfully, IMO) do not answer questions about course content during the exam window, so if you wait to start studying until after the exam has opened, you may find yourself unable to get help if you're stuck.

Finally, the instructors/TAs for this course were excellent. They were readily available to assist students, from Piazza to Slack to Office hours. They were courteous and respectful to students, extremely fast with grade turnarounds, and overall ran the course better than just about any class I've taken so far. No mid-assignment revisions, no absurd grading policies, no "gotchas".

Warning about previous reviews: I found many of the claims in previous reviews to be INACCURATE: that the exams didn't test student's understanding, that the exams didn't look like the practice exams, that the assignments lacked specifics, that the lectures didn't help with the assignments, or that the documentations of libraries/frameworks/tools for the assignments were lacking.

The exams and assignments' questions were among the best quality in all of the classes that I've taken so far. Mid-term and final exams tested concepts from the lectures, were straightforward and fair. There were no trick questions. If you paid attention during lectures, took notes, and spent the time to understand the concepts, you would do exceedingly well (I scored in the 90%s for both mid-term and final; it can be done!). In fact, for a couple questions in the final exam, the answers were directly in the lecture notes, and so you could literally just copy and paste!

The lectures were excellent and provided all of the (basic) fundamentals to work on the assignments, and you would fill in any gaps by reading the required readings mentioned in the assignment instructions. Again, if you took notes during lectures, and did the readings, then the assignments would feel straightforward. This doesn't mean that they were easy, but at least you wouldn't get entangled in the concepts, and would instead be able to tackle the "puzzle" head on. The documentations on the libraries/frameworks/tools were sufficient and clear.

In summary, this is an excellent class on static and dynamic analyses. In addition to the application, it covers theoretical aspects: does an analysis terminate; is it complete; is it sound; can we do better, and if so, what the trade-offs would be? You will learn a ton, and end up with a deeper appreciation of static/dynamic analyses--the sort of things compilers and IDEs do to optimize/analyze your code. Also, Dr. Pryby and the TAs were responsive, fair, helpful, and awesome.

This class was pretty awful. It is the first class I had taken in the OMSCS program and it was a horrible experience. The lectures were ambiguous and maybe 2 of the 11 lectures helped with the assignments. The exams were also terrible. The practice exams they give you are not even close to what the actual exams are. They contain alot of concept knowledge in which the lectures do not go over. For the most part, the assignments are doable, but takes a lot of time and hitting yourself in the head. The exams are tough.

Automated Testing, Statistical Debugging, Datalog. Here's a testing program using java 1. 7 with terrible documentation written in sanskrit. Five of the assignments are easy. Two will destroy you. Check out chord. jar and Datalog before taking the class. They changed the course this semester to put the 2 hard ones in the middle of the semester, and every assignment after was so easy, I thought I was completely missing part of the assignment. The theories and ideas are good, some of the tools used are old.

This class is basically using tools that only grad students use which makes it hard to google. I was able to ask lots of questions and make my way through the course. It did introduce me to some pretty cool testing tools for Android but the other tools were a pain and will probably never be used again. I would recommend this course as it does add value.

Grading of the assignments and exams is not to test the student's understanding of the material, but to somehow make the students drop the course. The assignment questions are much advanced from what is being taught in the lectures and there is few material provided with the intention to teach what the students are supposed to understand from the course. This course goes against the ethics of teaching itself. I fared well in certain assignments because I already had thorough understanding of the matter under test, which I had gained from my own previous work experience.

The outdated APIs used for some assignments themselves need much workload to decipher, which will not find much practical use in a real-life scenario. Even after putting tremendous effort, many things remain unclear and there is no where to turn to learn, other than depend upon the understanding of fellow students who themselves are struggling.

Kindly think twice before registering to this course if you don't know the entire subject matter beforehand, please don't expect to learn anything new from here as nothing is really 'taught' here.

Course seemed like a "phoned in" course to me. Neither TA's nor professors were very helpful with troubleshooting issues with provided VM, setup tips came from other students. Assignments lacke specifics in a lot of cases, making them harder than they should have been. Further, the two middle assignments were based on depricated APIs with not much in terms of information available on them, making those assignment even harder. The assignments and class lectures loosely fit together, but not well. As for the exams, the mid-term was open book, but the material on the actual midterm exam did not match that covered in the provided practice midterm, nor in the office hours "midterm review" sessions. Granted, you were given 3 hrs to complete the midterm, but when the average grade was somewhere between 27-32 out of 50, something is just not right. I decided to drop the course after lack-luster performance on the midterm as well as several assignments, also after asking professor / TAs for help or clarification. The assignments and midterm were returned with virtually no explanation of what was done wrong, making learning from mistakes difficult at best. Further, when approached, the TAs / professor gave generic answers that did not really help much. My advice, proceed with caution if you choose to take this course. Avoid it if you can, its not worth the time investment or hassle.

For NON-traditional students – the course is not difficult but expects you to be familiar with many different tools. Don’t get caught because you not familiar with basic linux debugging for java and c.

In a sentence, the material is interesting but assignments' docs too suck and staff too sour to enjoy course much. Still, recommended if keen interest in the topic.

Re: my ratings, I attempt to succinctly capture every detail from the lectures, so I end up spending a lot of time on the OMSCS courses, including this one.

I found most of the assignments really difficult, mostly because the documentation provided with the software was bad and, strangely, there were no teacher-provided materials (other than scattershot Q&A) towards filling that in. Quite too much sleep-deprived hacking resulted.

On the other hand, the exams were well written and the practice exams helpful. I thought the video lectures also very well done. I didn't care much for the teaching interactions though, because they were aloof, with all that implies. Fellow students were more helpful (as is often the case).

I think that of all the courses that I have taken in the OMSCS program this is the one that I disliked the most. The course material presented did not prepare you in any way for the mid term. The material was informative to an extent but could be much more useful. The TAs seems to not be very helpful as well. Now that may have changed now. All in all I think that they need to take a look at this course and determine if it is something that they would like to teach at all. If it is something that they could teach maybe it could be updated to maybe at least follow a single textbook.

This was my first summer class, and it was perfect for it. There was a lot of material crammed into a few weeks, but I felt the amount and complexity were fair for the time.

The field of analysis and testing was pretty new to me, so I learned a whole lot of new methods and techniques. Every lesson was about something new and exciting that we got to apply right away on one of the multiple assignments we had. It is a very hands-on class, so be ready to use Java and have a deadline almost every week.

The two exams were open notes. I had to put a lot of hours studying the material and I fared very well on both. The questions were fair and covered the content of the lessons, but you have to make sure to know it all, albeit of having the notes in front of you.

Overall: I enjoyed this class and the material it covered. It had more of a "testing in theory" instead of "testing in practice" approach that I found refreshing. I didn't find the class too difficult and it served nicely as a companion course in a semester where I needed to take two classes.

What I enjoyed: I found the material covered in the course quite interesting especially in the second half of the course. The majority of the assignments mapped to the material quite nicely with the exceptions listed below. The tests were somewhat difficult but really helped solidify my understanding of the material instead of being an exercise in memorization. The TAs (and other students) were helpful on Piazza.

My concerns: The first assignment didn't align well to the material covered in the lecture. It's not a major issue but it wasn't clear what they really wanted us to get out of the material based on what was on the exam. The last two assignments were too easy and didn't feel like they required an understanding of the material to complete. The last lesson (dynamic-symbolic execution) didn't have an associated assignment which was disappointing. The lesson on type systems didn't have an associated assignment.

Concerns I heard from others: The class sometimes requires you to use outdated APIs that are no longer supported/poorly documented. I agree with this but it wasn't a major headache as I've had similar experiences with other classes. Just know what you're getting into. The APIs are difficult to get set up. I used the class VM and kept up on Piazza so this wasn't an issue for me. Most of the material was recycled from a course offered by Stanford. I think that the material/assignments can be freshened up a bit per my comments above but this doesn't bother me. Lack of involvement by the professors. I agree but I think this is normal for many other classes I've taken. The TAs were helpful and the profs were active on Piazza prior to exams.

This class is exemplary of Tech. I was an undergrad here. Trust me, I would know.

"You will use my outdated Java API and it will suck your soul out. " "But sir, there're at least 5 libraries that do what yours does and are actually maintained... " "You will use my API. " "But sir... " "Welcome to Tech. " "Ok"

"Ok": the 4 year cliffhanger and counting. In all seriousness though, this class actually manages to decimate intellectual curiousity... and for that I suppose a round of applause is in order.

Edit: Mod I understand you stripping out ad hominem but please don't go all social justice warrior on my post or remove bits that are part of the argument. Really when you think about it, the fact that so many professors at this school are self ingratiated is actually part of the reason why classes like this, that are so poor and devolved from reality, exist in the first place

I felt this was the worse class I have taken thus far.

First the positives: Michael and Ben ran the class and did a good job keeping up with needs as well as having office hours. They attempted to be fair for the most part and I appreciate their efforts.

Now for the negatives. First problem, completely non-existent professor. He never showed up once and coming from a class last semester where the professor was heavily involved this was a let down. The videos were also weak in a number of areas where complicated concepts were glossed over and never really explained in great detail leaving the TAs to have to try and explain these concepts. The class also seemed to have copied in large part the material from a Stanford course CS295. Some of the assignments for example were taken word for word from the Stanford assignments. Luckily some of the videos from Stanford explain the material way better. I find this really disgusting considering we are always asked not to plagiarize.

Overall the material was not what I was expecting. A number of assignments used libraries with little or no documentation that was often times inaccurate or was utilizing libraries on projects that had been abandoned years before. I really think this class could be better described as to what it is covering and better if some of the more useless aspects were removed. There was some good material but it was mostly hidden away behind all the useless items I had to learn about.

I'm writing my review near the end of the summer semester, with only one assignment and a final left to go. From reading some of the other reviews I am wondering if this course was changed after last semester. I do not find it anywhere near as hard as some reviews have indicated.

I've been able to get by with about 3-4 hours per week of total time, during the summer session, and have a very good grade so far. The TAs have posted statistics and the median grade of the rest of the class is also very strong. I believe the midterm average was around an 80%.

The material and assignments are interesting. I'd actually appreciate a slightly deeper dive into some of the areas, rather than the pretty quick surface study that is given in the lectures. That said, the lectures are good, and there is nothing stoping students from researching more on their own.

The Prof has been mostly absent from piazza all semester, but the team of TAs have done an excellent job grading and responding to posts in a timely manor. I did not attend any office hours so cannot comment on them directly, but based on many posts I saw on the forum I gather the office hours were helpful for those that needed it.

This was a very tough class, but there were both pros and cons.

Assignments - aside from the last assignment (which i simply couldn't complete even after ~40 hours), they were all very well done and a bit fun. Some take an insane amount of time to complete, while others are very easy. The overall balance felt about right for the assignments.

Udacity - Some lectures were better than others, but the quizzes in them were excellent. They really helped you learn the material. Very well thought out and the best quizzes out of any class I've taken.

Exams - horrendous. Absolutely did not test our knoweldge of the material whatsoever. Questions came out of complete left field and I feel are more ego-driven to show off how smart the professor is and his vast amount of knoweldge outside of Software testing than to actually test us on what we know. I failed the final and got a D on the midterm. But then again, the median grade for each was in the D/F range -- this alone speaks volumes.

Time spent - This is hard to calculate, as some weeks were 40 hours, and then there were stretches of time where there was no work whatsoever. It fluctuates a lot, but I would say each assignment is 10-15 hours on average, and there were 8 assignments.

A word of warning - this professor is absolutely obsessed with finding and punishing people who copy or reference code from other students or past classes. Multiple announcements were made throughout the semester, warning students of getting an F in the class and using heavy threats demanding "they know who cheated, but if the student turns themselves in the punishment will be far less severe. " The constant threats were irritating and seemed to really negatively affect student morale, especially the honest ones. This was very unprofessional and should have been handled discretely without obnoxiously threatening the entire class.

All in all. It was a solid class with a good amount of work. Grading was generous, even with the impossible exams.

In short: I don’t recommend taking this class.

Please feel free to refer to Udacity videos or Piazza to verify my comments if you like. I tried my best to be fair & hopefully it helps others. I got a B in this class.

Content: Videos are highly analytic & professor often uses “so” & “therefore” to skip explaining a concept. I took this class under the impression that it’s related to the standard testing/QA but it was mostly about compiler design/efficiency. Video lectures quality is overall low especially towards the end. Pre-requisites: Pre-reqs were removed from the course page later on. You need to be proficient in Java, C, discrete math … Exams: I’ll summarize it with this epic story: Instructor posted a pdf with sample questions prior to the midterm & based on the comments on Piazza, majority of the class simply thought that he posted the wrong pdf or in the wrong class. Average for the midterm exam was in mid-50. Hopefully you can understand the gap here Assignments: They are written in a very dense academic format. Professor & the TAs weren’t helpful in explaining the assignment, providing resources or answering Piazza questions. They run a plagiarism program over all the assignments & you can end up at the dean office or get an F if they find you plagiarized. Piazza: Professor & TAs are very detached from Piazza but you see lots of helpful comment from the top 5% of the class. Anon post are not allowed. Professor/TA: In a nutshell I think I’d have enjoyed this class a lot more if we had an active/engaged, professional & experienced professor. The staff was not open to any suggestion or feedback. I felt they were very much used to the on-campus traditional style of teaching & communication with students. Professor Mayur only showed up to say the harsh words or threaten students. Before finals week he sent a mass email saying: “I find that you plagiarized in this assignment while grading, the consequences will be severe: an F grade in this entire course. ”

The class took a total U turn after midterms. The assignments before midterm were fun and were about topics that could be used practically like automatic unit test code generation, random testing etc. The midterms were open book but took many by surprise and it resulted in very low mean and median scores. The class also had a month long break after midterms were nothing was due.

The last two assignments were crazy difficult and people struggled a lot. There was not much help from the instructors and the Piazza forum was ran by only handful of students. The software analysis tool used was the one developed by the professor and it had limited documentation and no practical value unless you are into system programming, IDE development, Compiler design etc.

Software Test part of this course is good and has some practical use. Software Analysis is good to know and is explained well but not applicable to everyone.

If you are just looking to add one course to complete the course requirements, stay away from it. If you are not specializing in systems, stay away from it.

Content - The Udacity videos did a good job of presenting the course information. They presented the information in a pretty traditional, straightforward way (as opposed to the "socratic" approach of a class like ML), but they were sufficient. The quizzes embedded in the lectures were well-designed and more than just a trivial "pick the right answer".

Projects - Favorite part of the class. Each project gave us a chance to use a tool in order to get hands-on experience with a topic covered in the lectures. Most of these tools required (or recommended) Linux in order to get consistent results. It would have been nice to start with a standard VM image, but it's also not difficult to set one up yourself. Regardless -- the projects were well-designed, relevant to the lectures, interesting, and most of all -- fun!

Exams - Both exams were open-notes. You could even use PDFs on your computer during the exam. Therefore, the questions on the exams required less rote memorization and more active application of the concepts learned. From what I've heard, most of the students that had trouble with the exams actually struggled more with ProctorTrack than anything else. That's unfortunate, but not really a knock on the exams themselves. The questions were fair and did a good job evaluating comprehension of the material.

Administration - The TAs and the professor were very active and helpful on Piazza. This is not necessarily always the case in other OMSCS classes, so it was definitely appreciated here. Most of the grading was done in a timely fashion, although a few of the assignments took longer than average.

Overall - This class is an overview class -- you'll cover a lot of different topics, without going too far in-depth. The class is similar to CS 6035 in that regard. You'll get exposed to a lot of interesting ideas and use a variety of tools to explore those ideas, which hopefully encourage you to learn more in the future. I highly recommend this course.

This class requires you to setup and use many tools on your workstation and/or a VM. For Windows users, the setup I recommend is as follows in the host OS: (1) Visual Studio Community Edition (or higher) with Dafny extension, (2) JDK 1. 8, and (3) Android SDK. Then, create a Ubuntu VM with the following: (1) JDK 1. 7. Yes this is a lot of tools and you will need them all :)

The assignments are for the most part pretty interesting! There is usually a "trick" involved and once you figured it out things will just click. However, plan on some extra time for the static analysis assignments towards the end of the class. Some of the students spent hours and hours on the last assignment, (Constraint Based Analysis), and the solution is surprisingly concise at about 20 lines of code.

The midterm and final were pretty difficult. On the midterm there was a generous curve and the instructors said that there will be a similar one for the final. The material comes from the lectures and is expanded upon. I would recommend watching each lecture twice; once before doing the corresponding homework and once before the test. Also, plan on re-taking quizzes using pencil and paper when watching for the second time (since you can't blank out previous submissions on Udacity).

This is a fun and great course. I'm happy to see more and more courses going online so that OMSer's have more choices. I don't come from a CS background and took this course mostly for curiosity and the fun of puzzle solving. It's eye opening to see so many techniques, each with their own benefits and trade-offs. The lectures are not the most entertaining ones, but the organization is very good and I had no trouble following through. The assignments and exams focus heavily on application of concepts and problem solving.

Each assignment involves reading the documentation on your own. It sounds intimidating but once you get your hands dirty with some examples, the problems aren't impossible. You can even get away with reading minimal amount of documentation. I'm not sure if the tools you learn in the assignments will be used in your future career, but they do serve the purpose of helping you learn. You'll need some knowledge about Java and at least be able to debug pointer related errors. The exams are difficult but open notes. It involves some tricks and a lot of thinking.

Regarding plagiarism, the solution as simple as "don't do it". There's no need to worry that your one-sentence code will most likely be the same as someone else'. The teaching staff know well enough about software testing to avoid false positives. Anti-plagiarism is not your enemy unless you are happy to graduate with students who cheat their way.

Full disclosure, I have no idea if I'm passing this course with a B or not, so I may not even get credit for taking it. Despite that, I learned a lot in this course. I'd even go as far as to say that this course actually prepared me for a separate assignment in another course which helped me apply what I was learning while taking this one. As a qualification, I am of the student build that didn't come with a complete CS background, instead having a degree in IT with an emphasis in programming. However at time of taking this course I have never taken a traditional algorithm and data structure course which the class expects you to come with some knowledge.

The course surveys different testing tools and ways of deriving analysis. It requires a lot of exploration and research on the student's part which I think is largely why there are many negative reviews because in many instances you were stuck reading through pages of a specific API in order to complete one part of the assignment. For me at least I can say that this has largely made me a better programmer though it feels like a trial by fire. We had a pretty active piazza environment so I was able to make some pretty good strides. However, be aware that the instructor is a stickler for plagiarism, so absolutely it is to your benefit to write your own code by yourself.

The tests are open book, but it doesn't really help because it will be based more on your ability to apply the knowledge to problems.

Closing statements, this was the first iteration of the course online so there were a couple of speed bumps. I expect that the course will get better with time.

Reading others reviews sounds like many people hated this class [like me]. In summary I do not recommend taking this class no matter what your specialization is. Do yourself, your family, friends and coworkers a favor and drop it. Also this class isn't really related to software testing and analysis, they took the concept from Stanford CS243: Program Analysis and Optimization and few other compiler design classes and made it up.

Hands down the worst class that I've ever taken in my life. The material isn't that difficult but the professor is just an amateur instructor.

This class was next to useless. There is such a broad overview of so many different testing strategies and techniques that you never actually learn anything in depth about any one specific thing. Additionally, the lectures are extremely boring and only contain small tidbits of information needed for the assignments and tests, they're all very long, and there are also a lot of them. It took way too long to get feedback for assignments and there was literally a month long break in the middle of the semester.

This was one of the more difficult classes I've ever taken. It's the only time I can recall ever having to just completely give up on getting an assignment to work and just turned in something I know didn't work. A majority of the assignments were not too difficult and only took a few hours. On the other hand, the ones that were harder than average seemed almost impossible to complete. The lectures were very well done and he gives you just about everything you need to know to trudge and sweat your way through the exams. (The exams were open note... but they really aren't useful except to try to confirm some basic facts you aren't 100% sure of. ) That's because most of the time on the exam will be spent trying to decipher most of the questions that seem to be written as a brain teaser than an exam question. If the exam questions were more straightforward, I would be happier with this class overall. Do not try to cheat your way through this class. He will run your code through a plagarism detection system (MOSS). I didn't cheat, but it still stressed me out a lot that there were announcements saying people who cheated should confess or be reported to the dean. I would always worry that the system might flag my submission as a false positive when it was analyzed. This is probably the second most stressful class I've taken in the program. If I knew what I was getting into with this one, I probably would have just taken something else. The topics covered were far from what I was expecting. I wish they would break this into 2 classes... One for testing and one for analysis. I don't think I gained much personal benefit from the analysis part and very little useful knowledge from the testing part... Overall, a thumbs down from me. The biggest positive right now is that I took the final yesterday and I am just so happy that is over and done!

This was a fairly interesting class, with a few serious flaws. The teaching style of the professor was somewhat detached and aloof. He rarely showed up in Piazza, and left things mainly to students. The head TA was active, but his style was also a problem, usually answering questions with more complicated responses that did little to address the fundamental problems that students were having.

The assignments were written in a very dense academic style that made for a lot of confusion in the students.

The assignments were very challenging, with the mean scores on the low side. The assignments often required considerable outside research. In some cases we were referred to external resources and documentation that didn't match the stuff provided with the assignments.

The first assignment was on a tool called Dafny, and it wasn't covered in class at all. We were given the assignment with a brief overview, referred to the Dafny website, and left to puzzle it out on our own with virtually no instructor or TA guidance.

The midterm was very challenging, and didn't have enough time allocated for it.

In spite of all of this, the material was genuinely interesting. This could be a fantastic course if the problems were addressed.

Overall the class covers some neat topics. Assignments are directly related to the lectures and utilizes the tools discussed in the lectures. In fact, some of what we've done I would like to see used in my job.

Assignments aren't too difficult and aren't too time consuming. Some will require extra reading but overall they should only take a few hours to finish. Exams were difficult, in part due to the exam format. ProctorTrack did make some updates for the final but the format still was kind of wonky (making table with text is kind of hard, long questions were in a scrollable box so you had to keep scrolling back and forth to get the full picture). There wasn't anything in the exam that wasn't covered in the lectures, you just need to make sure you have a real understanding of the lecture material and really understand the solutions to the lecture quizzes.

There are some pacing problems, but that will likely be improved in future iterations of the course. Up to the midterm there's something to do every week, but after there's not much material left to cover.

Really interesting class - it does a good job of balancing practical and theoretical aspects of software analysis, and covers some very interesting tools and techniques. The videos are very well done, and explain the concepts well.

It suffers from some pacing issues - it switches from being very busy to having nothing to do for a few weeks - but I'm attributing that more to it being the first offering of the class.

Recommended.

Excellent presentation, clear and concise lectures Very good overview over modern, academic approaches Expands thinking about testing and offers a glimpse into academic advances Quizzes in lectures really help understanding the content Very well paced, consistent amount of work every week Great choice of topics, good little insights into previous and current research Very active professor and TAs Generous grading curve

The assignments and lectures are very informative and interesting. However, the mid-term was very tough. Coupled with the horrible interface of proctortrack and the questions, many of which were not covered in the lectures and the limited time given for the test will result in a bad score for me at least. Professor said he will grade leniently. We also have a comprehensive final which I am not looking forward to. The exams in total make up 45% of your grade.

• 20% Midterm
• 25% Final
• 50% Assignments (8)
• 5% Participation

Just took the midterm, which was proctored (ProctorTrak). It was open notes, where you could view slides with a PDF viewer. The questions mirrored the quizzes in the lectures. Exam format was multiple-choice, multiple-select, free-response.

The pace is approximately 1 lecture per week with 1 assignment every 1-2 weeks.

Assignments so far have paralleled the lecture topics, covering Dafny, Korat, Randoop, Statistical Debugging, Delta Debugging, and Dataflow Analysis. They are interesting and graded pretty quickly.

The instructor is excellent and very active in Piazza. The material is very relevant. Highly recommend.

Mid-semester review, for those looking at this course for Summer/Fall. I would highly recommend it! I think it would also be very doable as a Summer offering. The course is currently structured with half of the course grade coming from projects (some due within one week) and half coming from proctored tests.

The projects involve setting up and using a software analysis toolkit. These can sometimes be time consuming to setup, but generally take 5-10 hours at most to complete. The ones requiring code are in Java (so far) but you should also be able to read C. (two I haven't seen yet)

Tests are fairly straightforward, mainly coming from the quizzes in the lectures, which are high quality. The tests are long, and it was not uncommon in the midterm for students to work up to the time limit. They are open note.

If possible, take CCA and or InfoSec prior to this class. Lots of overlap, and you will be very comfortable with the theoretical material here if you have taken CCA prior.

First of all, this is a mid sem review by request so poke me if I don't update after...

This class is a very interesting and a ton of fun! It can be frustrating and the projects range from very easy to medium, but no where near as hard as IOS for example. This was the first sem of the class, so there are some bugs to work out, interesting wording, etc. The office hours haven't been great, but that is my only complaint so far. You do not need to be running native linux as the course description might suggest. I didn't need my Ubuntu VM (same one I used for IOS) until the 5th assignment and some students got it working in Windows I think. The lectures are well done and interesting. I highly recommend this course! (so far)

It doesn't hurt to know a little C and Java before the course, otherwise it isn't a bad beginner course.

Assignments are all individual.