Once per year, now in the third quarter of the year, the User Support Department of ESO launches a Paranal Service Mode User Satisfaction Survey campaign.  This report details the findings of the September 2016 survey campaign, while previous such reports are found here.

We view these reports as an important way to close the loop with the ESO Community, to gather feedback on issues that need to be addressed, to thank all respondents, and to demonstrate that such feedback is important to us. To this end, here we provide a summary of the responses received, predominantly in the form of graphs. It should also be stressed that for those cases where respondents did identify themselves and did make specific free-text comments we have contacted them by e-mail to address their particular comments.

Methodology and General Results

The ESO Service Mode Questionnaire is always available on-line for users to fill in but the usual rate of return is less than 2 per month. However, experience shows that a targeted campaign of asking users to fill in the survey results in many more survey completions.

In September 2016, we again took this approach, and asked Principal Investigators (PIs) of Service Mode runs scheduled for Paranal in Periods 97 and/or 98¹ (plus their then-active Phase 2 delegates) to complete the survey by a fixed deadline. We thus solicited a response from 457 PIs and their then-active Phase 2 delegates (189 individuals). Because of overlap this amounts to a total of 522 individuals which were contacted via e-mail. A deadline was set for two weeks from the date of contact.

A total of 121 responses were received by the deadline, at which time the deadline was extended for an additional week.  After that additional week the number of submissions had risen to a total of 187 (some 25 of which were not fully complete), representing a 35.8% response rate. This rate is our highest to date, and we sincerely thank all of you who participated in this!

As a start in detailing the results from the survey, in the figure below we show the number of responses we received per instrument. In spite of the overall good response rate the large number of instruments offered in Service Mode means that on average we received about 14.5 responses per instrument.

In the following three pie charts we present a general overview of user satisfaction (in percentage of responses) with three general items:

• the help/advice provided by the User Support Department during the Phase 2 process
• the Phase 2 web documentation, and
• the overall support provided by the User Support Department.

Note that the sum of the responses to the question about one’s satisfaction with the help/advice provided is less than the total number of survey responses. However, there is no reason to expect, a priori, that these two numbers should be equal. This is because the responses to this question come from a subset of respondents (those that actually received help/advice at Phase 2), each of which may have received help/advice from multiple instruments. 

Seeking Help, Run Information, and Run Problem Resolution

Amongst the respondents 23.5% indicated that they had contacted ESO for non-Phase 2 related reasons within the previous 12 months (about the same percentage as in the September 2015 survey). Of these 44 respondents, 39 contacted ESO via an e-mail to usd-help@eso.org (again consistent with September 2015), with the remainder distributed between other methods (e.g. clicking on ‘Ask for help’ within the ESO User Portal).  Below we show the degree to which these respondents were satisfied with various aspects of the resulting exchange with ESO.

Some 71% of the respondents checked on-line for information regarding the progress of their observational programmes (this represents a marked increase over the 58% of respondents that reported having checked this information in September 2015).  The survey asked those that did check for that information how much they agreed with four statements about that information. The outcome of those questions is shown below.

Finally, we asked if the survey participant was alerted to any problems with their Service Mode observations during the period. For those that said that they had been contacted (53) we then asked to what extent they agreed with two statements describing the contact and the problem resolution. The answers are shown below.

P2PP and Other Observation Preparation Tools

Below, we show details of the feedback received on different aspects of the Phase 2 Proposal Preparation tool (P2PP) and other, instrument-specific, observation preparation tools. 

Since the number of responses per observing preparation tool other than P2PP is rather limited (see the table below), any presentation of individual-tool responses on documentation, ease of use, or functionality would suffer from small number statistics. Thus, in the figures below answers for all tools are combined. 

Finally, we asked survey participants the question, “How satisfied are you with the ETCs you have used?” The responses are shown below.

Notes:

¹The total time allocated in Service Mode for Periods 97 and 98 was 11458.2 hours, while the corresponding number for Visitor Mode was 3965.4 hours.  Thus, the September 2016 survey targetted PIs (and their then-active delegates) representing 74.3% of the total VLT/VLTI time allocation, including all public surveys.