Surveys and Statistics

Everybody Welcome Online Surveys - the role of the agent

The main aim of these surveys is to get some national handle on four key questions:

1. What proportion of churches have gone online supporting what proportion of churchgoers?
2. What proportion of churchgoers are not accessing online church?
3. How does online ‘attendance’ compare with in-person pre lockdown?
4. So how many new ‘attenders’ do we think there are?

The method is to find as many ‘agents’ as possible in a variety of settings to conduct small scale surveys of all the churches in an area or grouping they know quite well. The key is to get a near 100% response rate, otherwise we will never know the answer to Q1 as those going online may be more likely to answer a survey and the sample will not be random.

To this end, the questionnaire is as simple as I can make it and churches not offering online services simply need to give the agent their name and say ‘no’. The local group of churches should also find helpful interest in the results of their own survey and comparison with others.

Examples of groups would be Anglican Deaneries, Methodist Churches Together groups, X Denomination churches in Town or City Y etc. The best size of group would be 10-30 churches. Keep the size manageable to keep the agent’s job within bounds, allowing focus on 100% responses.

We are developing a simple calculator to convert measures such as ‘peak devices’ and ‘views’ into estimates of people accessing services.

The agent should email the questionnaire to a contact for every church on their list with a request to email the completed answer to them within a given time, say a week. The agent would follow up non-returns, perhaps with a phone call. When the agent has got all or nearly all responses they should email every form to [email protected].

We will then analyse, add to our database, and send the agent the results.

The agent should send out the questionnaire as a ‘word’ attachment, asking for the completed form to be saved, named with the church’s name and returned as an attachment. Then the agent should send all their attachments to us in a single email so they stay together. This email should also name and list the group of churches, saying which, if any, churches failed to respond. Explanatory notes about, say, the complications of groupings of churches sharing a minister and online service would also be helpful.

If you are willing to be an agent email us at [email protected], and we will send you the survey form.

We’d like to get a better statistical handle on what is going on. Can you help us by surveying your group of churches?

Small scale surveys

Can you help build up the bigger picture by acting as an agent to look after a small-scale survey of your group or area of churches?

For example:

• A Church of England Deanery
• A Methodist Circuit
• Churches Together in X
• All Elim Churches in the County of Y

If you are up for this, email us at [email protected] and we will set you up with a questionnaire.

You may need to follow up non-responders as only a very high response rate will tell us what proportion of your set of churches has gone online.

How to calculate online ‘attendance’ from Facebook streaming and video

Warning: Only numbers geeks will enjoy this

If you are not a numbers geek, please at least look at the results and trust me!

Method summary:

1. Devices watching most of it live
2. Devices watching most of it on video
3. People per device
4. Attendance is 1 + 2 x 3
5. Compare with ‘in person’ last year

Results:

Attendance at St Lawrence Eyam (my church)

	2019	2020	% change
Palm Sunday	156	230	47%
Easter Sunday	221	310	40%
Low Sunday	98	150	53%

But remember that 2019 was a count and 2020 is a heroic estimate.

You may already have noticed that Eyam Church does better than the average village of 1000 souls, but we are the village that locked itself down in 1666 to contain the plague within itself. Despite social distancing, 260 of us died. I sometimes wonder whether the maxim that the church is built on the suffering of the saints and the blood of the martyrs applies to us.

Non-geeks can look away now.

Frankly, I’m quite proud of what follows. Please don’t find a hole in it. I’m an old man who has only simple pleasures remaining to him. However, I do suspect some clever young techno-wizard will soon tell me the three buttons to press to make Facebook yield the answer itself. Please let me know if that is you, but do it gently.

Method in detail:

Step 1: Live Participants.

Analytics tells you: Peak live Viewers (ie devices), length of service and total number of minutes viewed.

• For example, for one of our Easter Sunday services our Peak live was 77, length was 50 minutes and minutes viewed was 4344. Immediately there is a problem because 77x50 is only 3850. 4344 divided by 50 is 87. So did the counter miss some devices or exaggerate the total minutes or underestimate the length? In practice, I saw the devices monitor wobble up and down slightly all the time. I suspect some people did not manage all the service and some devices may have been hard to pick up. So I’m going with an estimate of 82 devices that were present for a good chunk of the service.
• In our example, the video length was still 50 minutes and the average view length was 2 mins 21 seconds, divided as follows:
• 901 devices average 141 seconds 901x141 = 127041 seconds
• 480 devices saw 3-10 seconds 480x6= -2880
• 216 devices saw 10-60 seconds 216x25= -5400
• 205 devices saw over a minute 118761 which is 580 seconds each, that is 9.7 minutes
• So roughly 200 people saw an average of about 10 minutes each.
• Then let’s say that we’ll count in anyone who watched at least half.
• Drop off is likely still be high in the early minutes then settle down.
• All we can do is make a reasonable guess at how many made 25 minutes.
• For example, the average of 9.7 minutes could be achieved by 20 devices watching all of it, 30 watching half and the rest watching just over a minute.
• Or by 15 watching all, 30 watching half and the rest watching an average of 3 minutes.
• I’ll be a bit cautious and guess that 40 devices out of the 901 followed enough of it to be counted in.
• Although we have discarded most of the views as casual glances we are left with enough to make a significant difference.

Step 2: Views.

Analytics tells you: number so far watching over 3 seconds, over 10 seconds and over 1 minute, together with video length and average watch time. By assigning plausible averages to the 3-10 second people and the 10-60 people (I use 6 seconds and 25 seconds) you can find out how much of the total view time is left for those who lasted over 1 minute. Total view time is found by multiplying the average by the total number of people.

Step 3: People per device.

Most live participants were on large devices. I was able to identify most of them from their comments and I know who lives alone or who lived as couples. We had a families (Messy) service at another time so hardly any device had more than two people. The split was even, so the average people per device was 1.5. For the families service it was 2.4. However, the video catch-up views were mainly on phones so count 1 person per view.

Step 4:

We have 82 x1.5 = 143 live participants + 40 x 1 = 40 video participants, making a total of 183.

• Being a cautious soul I round that down to 180 and add the 130 estimate for the other service achieved by the same method to get my 310 estimate for the whole morning.
• I then looked up from my records attendance the same Sunday last year and produced the table above.

Missing Regulars.

Like most congregations, we also have previously regular attenders who don’t possess a device or don’t know how to use it or don’t want to use it. These are mainly but not exclusively our older members. My best estimate is there are about 30 of them in our church, of whom you could expect to see about 25 on Easter Sunday and maybe 20 on other Sundays. Hopefully, when lockdown ends they will return in person.

However, how can we best keep in touch in the meantime? What can we do for them?

Bob Jackson