Security Stream Product Quality Assurance

Non-Technical Threads Talk to a SUSE Technical Services Manager
1

I guess this falls more into Michael Mychalczuk’s domain…

To be blunt, what the hell is going on?

Over the past 12 to 18 months we have noticed a drastic decline in the
quality of products released by NetIQ, specifically across the security
stream (IDM, Sentinel, Access Manager, eDirectory, etc). The change has
been noticable and dramatic, so something has changed or begun to fail
in a significant manner.

Alone I have 22 SR’s currently_open with the majority linked to a bug
of some description. The majority of those bugs are not related to some
bizarre way we’re using the product, but basic core functionality of the
product (I’m not going to put specific examples in this post, I consider
it a wider issue than that).

When Product Managers tell me that they have strong Testing and Quality
Assurance for their products, I just have to laugh. “Quality Assurance”
is supposed to mean that NetIQ assures the customer that the product is
of high quality and has been tested to behave as documented, which is
just simply not the case anymore.

Too much emphasis has been placed on shoehorning new features into
products with little to no validation that the core functionality has
not been compromised.

On a slight side note, the timing of these issues could not be worse.
The State Government is in the process of reviewing the ICT roadmap with
emphasis on the IDM, Access, and Security/Privilege components. As part
of this review, questions are being asked about product cost and
suitability. The process has already begun in earnest with various
vendors presenting their products. Having poor functioning products come
out of NetIQ is making other vendors look more attractive, despite NetIQ
being the incumbent.

It never used to be this bad, what has gone wrong?

–
Visit my ‘Website’ (https://www.isam.kiwi/) for links to Cool
Solution articles.

ScorpionSting’s Profile: https://forums.novell.com/member.php?userid=1663
View this thread: https://forums.novell.com/showthread.php?t=507491

2

Thank you for taking the time and posting your thoughts on the forum!
The first thing that is appropriate for me to do here, before going any
further, is to say that *on behalf of the teams I offer to you, and
everyone else that reads this post, my personal and our collective
apology. *

Speaking candidly for products that are part of the privilege portfolio
(CG, DRA, GPA, PAM), we have found some significant breakdowns in our
processes and our testing technologies. I’m happy to say that those
issues were addressed for the upcoming release of DRA, and that other
product teams are looking at similar exercises for their products in
their near-term release cycles as well.

Over the past 18 months, and a bit longer, there have in fact been a
number of changes going on behind the scenes within product engineering
and product management teams across the portfolio products. I want to
assure you, and others, that there has not been a conscious change made
to our testing procedures and personnel with the intent to reduce our
focus or emphasis on delivering high quality software. However, as you
have pointed out there has been a need to rapidly address some market
needs, and what has happened is that there have in fact been cases where
we simply did not test far enough in some areas.

In one of our product teams, we recently performed a comprehensive
end-to-end test case review that covered over 2,000 test cases. In that
analysis, we discovered that when we had added new functionality, we had
created both unit and functional test cases, but had not adequately
addressed regression test cases which test how our new functionality
interacts with existing functionality. This omission in effect created
several blind spots in our final testing phase.

Writing a response to you, and others, on a forum is easy. Let’s be
frank, words are free, easy, and available in plenty. The only thing
that will demonstrate our commitment to quality is our actions. I am
confident that in the releases around the privilege products that will
be occurring in the late June time frame this year you will see progress
in this area.

–
mychalczukm

mychalczukm’s Profile: https://forums.novell.com/member.php?userid=170230
View this thread: https://forums.novell.com/showthread.php?t=507491

3

Thanks for taking the time to respond Michael.

–
Visit my ‘Website’ (https://www.isam.kiwi/) for links to Cool
Solution articles.

ScorpionSting’s Profile: https://forums.novell.com/member.php?userid=1663
View this thread: https://forums.novell.com/showthread.php?t=507491