Building a trustworthy system that effectively protects the confidentiality, integrity, and accessibility of its data to meet information security requirements is mainly the purview of the developers. However, stakeholders (especially Subject Matter Experts, or “SMEs”) have an ethical responsibility as well. It’s possible for the SMEs to affect trustworthiness by placing constraints on the user experience ( UX) design contrary to good design practice. These constraints can adversely affect system integrity, degrading system trustworthiness. Furthermore, the rationale driving the constraint may be unethical from a business (and not technical) perspective.
Before I get too far, let’s take a moment to define trustworthiness. According to the National Institute of Standards (NIST) Special Publication 800-53 Rev 4 (NIST SP 800-53 Rev. 4), trustworthiness is defined as:
The degree to which an information system (including the information technology components that are used to build the system) can be expected to preserve the confidentiality, integrity, and availability of the information being processed, stored, or transmitted by the system across the full range of threats. A trustworthy information system is a system that is believed to be capable of operating within defined levels of risk despite the environmental disruptions, human errors, structural failures, and purposeful attacks that are expected to occur in its environment of operation.
The process for achieving the ATO includes evaluating a large number of security controls. Among these are the System and Information Integrity Control Family, specifically the controls having to do with information input validation. A technical term that may not resonate with many stakeholders, information input validation refers to how the system checks data for errors and handles erroneous inputs. In other words, making sure, to the extent possible, that bad data doesn’t get into the system and recovering from bad data it if should get in.
Most of us working on projects are held to certain ethical standards. The terms of employment where you work (if working for an ethical organization) include adhering to ethical standards. For example, when I worked at the US Environmental Protection Agency, I was expected to follow the ethical guidelines for federal government employees. To back this up, I was required to take ethics training every year. If I didn’t complete the training on time, EPA would suspend my access to its network until I finished it. That was quite an incentive and enough of a deterrent for me to always have completed my training by the deadline.
EPA also required that I take its scientific integrity refresher training every year (required under EPA’s policy on scientific integrity). I was one of the few (and, sometimes, the only) information technology specialist in a room surrounded by scientists and engineers. It might have seemed strange for me to be in the class (though my academic background is in science), but the systems I supported did collect and process scientific data used for making compliance decisions.
In addition to the yearly EPA ethics and science integrity training, I also belong to the Project Management Institute (PMI) and the Association of Computing Machinery (ACM). Both organizations have a set of professional guidelines that I’m expected to follow as a condition of my membership (PMI here, ACM here).
Ethical Design in a Complex Project Environment
Ensuring ethical design can become an issue for complex software development projects involving multiple enterprises. While most of these “user enterprises” are doing the same thing, they may have differences in business practices. Sometimes these differences are caused by interpretations of business rules, especially if there are ambiguities in those rules. Organizational structure, workforce availability, and other business environment factors can also cause differences between business processes amongst stakeholder enterprises.
Perhaps the biggest challenge facing the designers working on this kind of project is to interpret the intention of the business rules, clarify the ambiguities, and design business processes and data models that work for all stakeholder enterprises. Needless to say, that is a tall order. The best way for the project team to define the new processes and data is to work with the intended end users – the subject matter experts, or “SMEs”, to define the new, common business processes and data models and seek consensus.
Each SME brings baggage to the project. They’ve been brought-up professionally doing things a certain way (those business environmental factors I mentioned). Sometimes the SMEs don’t know why things are done the way they are except “that’s the way it’s always been done”. In the absence of a strong central authority, what often happens is each organization modifies the business processes to suite its own environmental factors. Over time, significant variance may develop between organizations, resulting in different versions of the same business process and data definitions. Differences can become so great that some of the stakeholder organizations may have business practices discouraged or even forbidden by another.
The trick for the business analysts and user experience (UX) designers is to come up with a design that remains true to the intended goal of the business processes. What can happen is a “damned if you do – damned if you don’t” scenario where some of the stakeholders are satisfied and others aren’t. There might be situations (and I’ve seen this) where a design solution acceptable to one set of users is unacceptable to another because of ingrained business practices (the “baggage” noted above).
Getting back to good design practice and ethics, another challenge is that the SMEs on these kinds of projects are usually not software development professionals. That does not mean they aren’t operating under a code of ethics. I’ll go out on a limb and say that SMEs are operating under their organization’s code of ethics which they are responsible for following. (Unless, of course, the SME is working for a criminal organization that has no code of ethics!) A disconnect happens when SMEs are unaware of (or fail to understand) the ethics of software system development. I’ll give an example of how this can happen on a software development project.
An Example of a Complex Project
The goal of this fictional project is to develop and deploy a new system for regulating an industry. Funding for and management of this project is the responsibility of a central organization that will provide the system to scores of regionally dispersed user regulatory organizations at no additional cost. Most of the end users are located in the regulatory organizations, the majority of which are independent from the central organization funding and managing the project (the “funding” organization).
While outnumbered by the sheer number of regional organization end users, funding organization end users play an important oversight role. There are two competing interests :
- Regulatory organizations doing business following their local procedures that are usually (but not always) consistent with the central organization’s expectations, and;
- Central organization monitoring the regulatory organization programs for consistency (oversight and auditing).
As the regulatory organizations are “where the rubber meets the road” the central organization needs their commitment to use the new system. But it must take care that it deploys a system that, while making life easier for the regulatory organizations, doesn’t undermine its program oversight responsibilities.
To facilitate input from the regulatory organizations, the project team enlisted the services of a Subject Matter Expert (SME), who, while not an information technology professional, has broad and deep business experience in their home regulatory organization. Remaining project team members include:
- A project manager (PM) supplied by a Project Management Office. The PM has knowledge of, but light experience in the regulatory program the new system will support and the political environment.
- A business analyst (BA) assigned to the project who works in the program office of the central organization. The BA is an IT professional experienced in supporting the regulatory program IT needs and is deeply aware of the politics and challenges of the program.
- A contractor development team managed by the PM described above. The team is using a development approach combining aspects of SCRUM and waterfall.
On this particular project, when designing the user interface, SMEs paid very close attention to mouse clicks. Like the old FORTRAN GOTO statement, excessive mouse clicks were considered bad. However, there was no explicit definition of the threshold for determining when mouse clicks became excessive. It was much like pornography; the user would know it when they saw it.
One of the reasons why mouse clicks became so important is the primitive user interface of a widely used legacy system. Users were forced to open and close numerous web pages to accomplish simple tasks and the on-screen controls were primitive, especially compared to today’s responsive webpage designs. The SME wanted to free the users from the shackles of the dated user experience of the old system. Furthermore, have a more streamlined user experience would make the system more compelling and acceptable to the other regulatory organizations.
During a design session, the team was discussing the interface for a feature to capture the results of a field inspection. Results of the field inspection are classified under a dozen categories. Within each category, there are five levels of severity starting at “no issues found” ranging up to “very severe” reflecting the most severe issue found for the category. To complicate matters, an inspection can cover some or all of the categories. So an inspection could have severity values for, say, eight categories while the remaining four categories were not evaluated.
While walking through a proposed design, the BA described how the default value for all category severity levels would be “not evaluated”. When entering the inspection results, the user would change the severity level for the pertinent category from “not evaluated” to the appropriate severity level. If the inspection did not include items from a category, the user would skip over the category, leaving the value “not evaluated” intact.
This did not sit well with the SME, who was concerned about the number of mouse clicks required for an inspection involving multiple categories. Instead, the SME insisted that the default value for each category should be “no issues found”. The SME expressed their concern that users would not accept the system because, in their experience, most inspections did not find any issues and covered all categories. In this situation, the user would have to set the severity level for each evaluated category, which meant excessive mouse clicks.
However, the BA pointed out there are situations, especially for large regulated entities, where the inspections required several visits over an extended period of time. The BA suggested an alternative workflow where, with a single click, the user could indicate that the inspection covered all categories and found no issues in any category. With this one click, the system would change the value for all categories from the default “not evaluated” to “no issues found”.
This compromise did not satisfy the SME, who continued to insist upon the “no issues found” default. The BA advised the SME that it would be improper to use “no issues found” as the default value because it was poor design practice. There reasons the BA gave included:
- The system is assuming the inspector reviewed items pertinent to every category (setting default values for all categories to “no issues found”) when they may have reviewed items for one or two categories.
- The organization must, within a certain timeframe, complete an inspection covering all categories for each regulated entity. For large entities, it often takes several site inspections to review items from all the categories. How can the system track completion if, by default, every category is evaluated by default?
- The proposed “one-click” solution would require the user to consciously verify (or consciously misrepresent, if so inclined) that no issues were found for all categories. The “click and save” solution would not take anymore time then clicking on the “save” button and accepting all of the default “no issues found” values as proposed by the SME.
- The central organization, through its oversight authority, has a responsibility to track the completion of the inspections over time. The completion rates are reported as program performance measures to the public. A careless user could easily undermine the integrity of the data used by the central organization for tracking inspection completion rates.
After a fair amount of discussion, the project manager acquiesced to the SME and proceeds with the “no issues found” default values.
An Ethical Dilemma?
When the system is evaluated as part of the certification and accreditation process, the review, if it knows of the default values, would likely identify it as a risk and recommend correction. However, because the system is complex and processes a large number of data inputs, it is just as likely that it would go undetected and released into production.
Is there an ethical violation here? My gut says yes because it gives the appearance that the system is “cutting corners” to speed data entry at the expense of data integrity. If I worked for the central organization auditing the regulating agencies and discovered the system enabled bad data, I would become very skeptical of the data quality. What other compromises did the project team make? And here we go…