Universal Questions - System Manager Guide

Introduction

The Universal Questions System provides Internet based questionnaires so that organisations can get feedback from their employees and customers.

Each person receives an email inviting them to answer the questionnaire. They click on a link in the email, and are taken through a simple process of logging in and answering each question. (Alternatively they can be given written instructions for accessing the system).

All administration of the system is done on-line on the Internet by the System Manager. The system makes this very straightforward.

The main functions of the System Manager are to:

Add, edit and preview the questions.
Add a run of the questionnaire. This specifies the name of the run, when it is to start and end, and which set of questions is to be used.
Send out the email invitations to complete the questionnaire. This includes the link for that questionnaire run.
Review and analyse the results on-line.
Export the results to Microsoft Excel for further analysis.

Access to the system

There are four types of user:

Anyone who is asked to complete a questionnaire.
System Managers who can view the results for their organisation.
System Managers who carry out all system administration for their organisation.
System Supervisors, who can add new organisations and authorise System Managers to access the system.

It is possible to log in by going to the home page and clicking on the link to log in. However it is far more convenient to automatically skip past those pages by clicking on a link. These links have the format described below.

A normal user logs in by clicking on a link like this:

www.universalquestions.com/login.php?client_id=123&run_id=456&run_password=abcdef
(where the client id is 123, the run id is 456 and the run password is abcdef).

This is the link that the System Manager sends out to the people who are being asked to respond.

A System Manager logs in by clicking on a link like this:

www.universalquestions.com/login.php?client_id=123&manager=manager
(where the client id is 123).

A System Supervisor logs in by clicking on a link like this:

www.universalquestions.com/login.php?manager=super

There are then three possibilities:

Registration is required for a normal user making their first visit to the system. They press the Register button and are asked for their personal details and can choose a personal password.
Normal login is used for a normal user revisiting the system, a secure run where the System Manager has already set up the users and assigned them personal passwords, and for System Managers and System Supervisors.
Anonymous login is automatically used for an anonymous questionnaire run. No questions are asked and the user can proceed directly to the questions.

System Manager Tasks - Maintaining the system and analysing results

Having logged in, the System Manager is given a menu offering three sets of options:

Maintain each of the tables of information that control the system: question, response, run, person, client.
Maintain sets of questions: create a question set from a template, copy a question set, delete a question set.
Analyse the results: who has responded, view the results, or prepare the results for export.

Each of these three sets of options is described below.

System Manager Tasks - Maintaining the tables

By going to the question table the questions can be set up, and by going to the run table a questionnaire run can be initiated. That is all that is required as a minimum, but this guide also explains the purpose of the other tables.

Each of the tables is maintained in the same way. There is a menu for each table with buttons allowing entries to be added, updated, copied, deleted, or viewed. A List All button shows all the entries in the table.

In each case the key of the entry to be worked on must be entered at the top. To find out what entries are already in the table, press the List All button, or to restrict the list, enter some of the key fields and then press the button. The View button also operates in a similar way, allowing the entries to be looked through in detail from start to end.

The key fields (except for email address) are all numeric, and can have up to 7 digits. The System Manager can decide how to allocate these numbers. For Question set and Run ID it is simplest to start with 1. For Question number it is recommended that it starts at 100 and then increments by 100, to allow plenty of space for adding questions in between existing ones.

Question table

The question table holds each question. The questions are grouped into "question sets". A questionnaire run will display each question in the specified question set, in order.

The question table menu has an extra button, "Preview". This is used to see what each question which has been entered will look like when the questionnaire is run.

Question type: There are 24 types of question. The pull down menu lists each type with a brief explanation. It shows the default headings for the multi-choice questions (these can be altered - see Custom heading below). It is a good idea to start each questionnaire with a type 1 and it is essential to end with a type 2. The best way to see what each type looks like is to use the Preview button.
Multiple choices: This applies only to the choice and multi-choice question types. It specifies how many rows are displayed. For a choice question, this is the number of choices. For a multi-choice question it is the number of question rows.
Mandatory: If Y the user is not allowed to leave the questionnaire page without answering the question(s).
Heading text: This text is displayed at the top below the run name and above the question text. It can be useful to repeat the same text across a group of related questions.
Question text: The text of the question. Where it is a multi-choice question, this is an introduction to the question and the individual rows have additional text.
Multi 1 text to Multi 12 text: The additional question text that is displayed on each row of a choice or multi-choice question. The number of rows is set by the value of "Multiple choices" above.
Custom heading: Y if a custom heading is to be displayed instead of the default headings, which are shown in the pull down menu for the question type. This applies to multi-choice question types 13 to 24 only. The first 2, 4, 6 or 8 column headings (depending on the question type) are replaced by text stored in the "Custom heading text" field (below).
Custom heading width: This allows the exact width of the columns to be set. The default is 65.
Custom heading text: If custom headings are being used then the text for all the headings is placed here. The text for each column heading must be separated from the next using a tilde character (~).
Comment allowed: Y if a comment field allowing the entry of a free text comment is to be displayed below the question. A comment field can be added to all question types except 1 (instructions), 2 (end of questionnaire), and 6 (free text).
Comment text: Text to be displayed above the comment field if one is displayed. This field also has a special purpose for the final question (question type 2). If it contains a website address (URL) then when the user presses the "Log Off" button they are taken to that website instead of the window being closed.
Validate number: Applies to number or date question types. Y if the input number is to be validated, . The minimum and maximum allowed are specified below. If it is a date question then the year is validated.
Validate low, Validate high: The minimum and maximum value allowed if "Validate number" is Y.
Jump question set, Jump question number, 1 to 12: Applies to the Choice question type only (type 10). If these fields for a particular row are not zero, and the user chooses that row as their response, then the next question displayed is the question set and question number specified. Important: The question set/question number combination MUST be greater then the question set/question number combination of the current question. In other words you can only jump down, not back up. (The flexibility of allowing a jump to a different question set allows the results analysis to be divided up easily.)
Jump default: Applies to all question types except type 2 (end of questionnaire). Y if the next question to be displayed is the question set and question number specified in the "Jump default question set" and "Jump default question number" fields below. Note that if this is a Choice question (type 10) and there is a jump caused by the response given (see "Jump question set" above) then that jump takes precedence. Example of use: this feature allows a jump around a set of questions that are not to be asked. These questions might be asked only if a Choice question had a specific response.
Jump default question set, Jump default question number: If "Jump default" is Y, then these fields specify which question is to be displayed next. Important: The question set/question number combination MUST be greater then the question set/question number combination of the current question. In other words you can only jump down, not back up.

Response table

There is normally no need to maintain this table. It has an entry for each response. To delete all responses for a run, go to Run table maintenance.

Run table

The run table has an entry for each questionnaire run. An entry must be added by the System Manager for each run.

Run name: This is displayed at the top of each page when the questionnaire is run.
Question set: This decides which set of questions are asked (see the question table).
Run start date and Run end date: Users are not allowed to do the questionnaire outside the specified dates.
Run password: The password for the run, which is required when the user logs in to the system. This is put in the link that is sent to the user.
Anyone login: This has three options, allowing anonymous (no login required), normal (recommended), and secure. For a secure run the System Manager must set up each user in the person table before they can log in (see the person table).

The run table menu has an extra button, "Delete responses". This is used to delete all responses associated with a run. This deletes all the results for that run. The number of records to be deleted is displayed and confirmation to proceed is required before they are deleted.

Person table

The person table has a entry for each person who has logged in (except for anonymous runs). The System Manager may need to maintain this table to:

Add users, or change their security level, for secure runs where only users set up by the System Manager are allowed to log in.
Add other System Managers who can only view the results.

The fields for each person are:

Title, First name, Last name: As entered by the user. For a secure run the System Manager enters them.
Personal password: As above.
Security level: The pull down menu offers all the possible levels. Note that a System Manager can add another System Manager (but not a System Supervisor).

Client table

This table is used by a System Supervisor to set up new organisations. There is no need for the System Manager to maintain this table and access is partly restricted.

The client table menu has extra buttons for a System Supervisor: "Delete questions", "Delete responses", "Delete runs" and "Delete people". These are used to delete all data associated with a client organisation. In each case the number of records to be deleted is displayed and confirmation to proceed is required before they are deleted.

System Manager Tasks - Maintaining sets of questions

Each individual question is added and updated by maintaining the question table, as described above. These options allow entire sets of questions to be added and deleted, without having to work through them one question at a time. In each case informational and warning messages are displayed to allow the user to confirm the copying or deletion that has been requested. There are three options:

Create a question set from a template

There are a number of templates which are documented on the page "Demonstrations and questionnaire templates" which is linked to from the Welcome page. Enter the number of the template and the number the new question set is to be given, and all the questions will be copied. They can then be tailored by maintaining the question table.

Copy a question set

An existing questionnaire can be copied and then modified, for example to tailor it for a different group of respondents. The original question set will need to be left alone as there are already results associated with it. Enter the number of the question set to be copied, and the number of the new question set. It is possible to limit the copying to a range of question numbers.

Delete a question set

When a question set is no longer required, it can be deleted. Enter the number of the question set to be deleted. It is possible to limit the deletion to a range of question numbers. Warning: once a question set is deleted, it cannot be recovered. No results can be obtained from questionnaire runs that were based on it.

System Manager Tasks - Analysing the results

There are three buttons on the System Manager menu for obtaining the results, "Respondents", "Results" and "Results for Export".

Respondents

The Respondents button produces a report showing who has responded, how many questionnaire pages they completed, when they completed them, and how many minutes they took to complete the questionnaire. This is a valuable tool for the System Manager as it helps identify who has failed to respond, so they can be chased. The report is output as a table that can be copied into Excel if required.

Results

The Results button runs a program which allows immediate detailed review and analysis of the results. A page is displayed offering a wide range of options, allowing the choice of what questions are to be reported on, whether detailed or summary information is required, and what detailed information is to be shown. It is easy to explore these options to obtain whatever information is required.

The analysis tables can be copied into Excel so that graphs can be produced, and some examples of how to do this are explained below under the heading "Results: Graphs from Excel".

Results for Export

The Results for Export button runs a program which prepares a table of the detailed responses, including scores, for export to Excel for further analysis. A page is displayed offering a wide range of options, allowing the choice of what questions are to be reported on, and what detailed information is to be shown. It provides instructions on how to copy the table into Excel, where the detailed responses can be analysed further.

Some examples of how to use Excel for further analysis are given below, under the heading "Results for Export: Analysis in Excel".

Reference numbers

When each question is displayed a reference number is displayed at the top right of the page. It looks like this: 1:100:10:M. The meaning of each part is as follows:

Question set.
Question number.
Question type.
M if a mandatory question, N if not mandatory.

The same reference number is shown against each question displayed by the Results program, making it easy to tie up the questions with the responses. If a user has a query, they can refer to the reference number to provide an unambiguous indication of which question they are looking at.

The reference number also forms the heading of each column in the Results for Export table. In this case it also shows the row within the page of questions, for multi-choice questions. So it looks like this: 1:200:16:4, or for a score: 1:200:16:4:S. The format is:

Question set.
Question number.
Question type.
Question row (for multi-choice questions only).
S for a column containing a score.

Scores

Multi-choice questions are automatically scored. The column one to the left of the centre scores +1, the column to the left of that +2, and so on. Positive scores indicate agreement with the question. The column one to the right of the centre scores -1, the column to the right of that -2, and so on. Negative scores indicate disagreement with the question. The "Not sure" and "Not applicable" responses score 0. The average score across all respondents for a question gives an immediate indication of the overall degree of agreement or disagreement.

Results: Graphs from Excel

The analysis tables produced by the Results program give an immediate view of the results. These can usefully be supplemented by graphs such as bar charts and pie charts. This section gives some examples showing how to achieve this using Excel. In each case the results analysis table (or part of it) must be selected, copied, and then pasted into an Excel spreadsheet.

Examples for a Choice question

Question type 10, Choice, allows the user to choose from one of a number of options. These might for example be which part of a company they work in, what their job function is, or how long they have used a product. The analysis table has three columns headed Response, Count and Percent. Under this is a row for each possible choice. Copy and paste the whole analysis table into Excel.

To see a bar chart of the counts by response, select the whole of the left two columns including the headings. Press the Chart Wizard button on the toolbar. Press Finish.

To see a pie chart of the percentages for each response, select the whole of the left column, then hold down the Ctrl key while also selecting the whole of the right column. Press the Chart Wizard button on the toolbar. Click on Pie in the list of chart types. Press Finish.

Example for a Multi-choice question (column of check boxes)

Question type 11, Multi-choice (column of check boxes), allows the user to tick any of a number of choices. These might for example be a number of areas that they have knowledge of. The analysis table has two columns headed Question and Ticked. Copy and paste the whole analysis table into Excel.

To see a bar chart of the number of people who ticked each question, select the whole table including the headings. Press the Chart Wizard button on the toolbar. Press Finish.

Examples for a Multi-choice question

Question types 12 to 24 all follow the same basic format. They are multi-choice questions with up to 12 rows. The analysis table has a first column headed Question, the second is headed Average Score, then up to 9 columns for possible responses.

To see a bar chart of the average score for each question, select the whole of the left two columns including the headings. Press the Chart Wizard button on the toolbar. Press Next. Select the Series tab at the top. Copy the data in the "Values" box (for the Question series) and paste it into the "Category (X) axis labels" box. Press the Remove button (to remove the Question series). Press Finish.

To see a pie chart of the percentage that selected each option for a particular row, first select the headings for each possible response. These are the headings of the table starting at the third column. Then hold down the Ctrl key while selecting the chosen row of percentage figures, starting in the third column. Press the Chart Wizard button on the toolbar. Click on Pie in the list of chart types. Press Finish.

Results for Export: Analysis in Excel

Having exported the detailed responses into Excel, a wide variety of tools are available for analysis.

AutoFilter

To analyse responses it is often helpful to apply a filter. To do this select the table and use the menu option Data/Filter/AutoFilter. Having applied selection criteria to one or more columns, one can select the responses or scores from another column.

This approach allows one to answer questions of the type "What do the Sales Managers who agree with the new bonus scheme think of the new car policy?". (i.e. filter on a job title question for "Sales Manager", also apply a custom setting filter for a positive score for a multi-choice question about the new bonus scheme, then select the column of data with the scores for the car policy question.)

PivotTable

Pivot tables provide a powerful analysis tool. Select the table and use the menu option Data/PivotTable. Press Finish.

A simple example would be where there are two Choice questions. Drag the reference number for one of them from the Field List to be a Row Field. Drag the other from the Field List to be a Column Field. Then drag the Email field to be a Data Item. The result is a table showing counts of how many people gave each combination of responses to the two Choice questions. For example if one question was which region they worked in and the second was their job function, the table would show how many people with each job function work in each region.

Return to the Welcome page