Comments from the authors in response to the questions from FAHCSIA

The numbers provided in Table 2 suggest a substantial increase in the turnover of

fruit and vegetables in the post intervention period compared to the pre-intervention
period (45%), yet this is not reflected in Table 4.

Box 2 presents the average per capita monthly sales and/ or volume of each of the
outcome measures. The first paragraph of the results section of the paper on p 551
provides a summary of these results. Box 2 presents the raw data. The values in the
Table do not account for underlying trends due to factors such as CPI, changes in
store practices, population growth or differences between stores. The per capita
figure is based on a population figure derived from the ABS 2006 census.

It is correct that a 45% increase in the monthly volume of fruit and vegetables sold
was shown for the post intervention period compared to a very low base in the pre-
intervention period. The Table in Box 4 shows (and this is also observable from the
figure in Box 3), that there was a statistically significant month to month increase in
the volume of fruit and vegetable sold in the 18 month period before the introduction
of income management. We have not examined why this month to month increase
occurred. It is important to note that it was modest at an average of 78g per person
per month. We have assumed that this would partly be due to population growth and
also to ongoing improvements in the food supply and retail practices supporting
better sales of fruit and vegetables. As there appeared to be a month to month
increase in sales and volume sold of fruit and vegetables, one would expect that the
average monthly sales and volume sold would have continued to increase over time
in the absence of any new interventions. Hence the increase in per capita volume
sold in the post intervention period compared to the pre-intervention period as shown
in Box 2.

In trend analysis what one looks for in determining the impact of an intervention is a
change in the underlying trend in association with the intervention (ie. a change in the
slope of the trend line). In relation to fruit and vegetable sales and turnover, our
results suggest that income management had no statistically significant effect on the
underlying trend. The rate in month to month sales and volume sold of soft drink
however did significantly change with the introduction of income management. Month
to month softdrink sales and turnover were constant in the 18 month pre-intervention
period (seen in the figure in Box 3). A statistically significant reduction in the month to
month rate of sales and volume sold was shown in the first 6 months of income
management. Thereafter, the rate in month to month volume sold increased
significantly at an average of 157ml per person per month. The rate of sales and
volume sold for nearly all commodities increased in association with the government
stimulus payment period.
Detail on the modelling used in the analysis.

The linear regression model used takes the form:

Yt = + 0*t + i*ti + p*tp + s*stimulus + c*storec

where

Yt is the value of the outcome variable at time t (measured from inclusion in the
study)

t is the time from inclusion in the study (in months)

ti is the time in the first six months of the intervention (in months)

tp is the time following the six month period following the six month period

stimulus is a binary indicator for the stimulus period and

storec is a binary indicator variable for store c (c=1..10, with one store omitted to form
the reference level)

An example of the relationship between the time and stimulus variables is given in
the table at the end of this document.

With this in mind, the coefficients have the following interpretation:

is a constant representing a form of average outcome

0 indicates the value of any overall linear trend over time

i indicates the change in overall trend in the six month period following the
intervention

p indicates the change in overall trend in the rest of the period following the
intervention

s indicates the change in outcome during the stimulus period

the {c} indicates differences between the stores.

If, for example, i or p was found to be statistically significant, we might conclude

2
that the intervention had led to a change in outcomes in the six-month period
following its introduction and thereafter.

The term fixed-effects model is used in a number of ways by econometricians and

statisticians. My guess is that the coefficients on the variables in the post-
intervention period shown in Table 4 are just reflective of change within the post-
intervention period. In other words the effect is not relative to the pre-intervention
period. Is that correct?

The coefficients relating to time following the intervention relate to changes in the
underlying linear trend and are therefore relative to the pre-intervention period.
(please refer to the table in the appendix).

How the variable for the pre-intervention trend was constructed (specified on page
550)? Is it a simple linear trend? Is it based on actual price data? Was this variable
used in any of the modelling for the actual physical volume of sales for example the
sale of fresh fruit and vegetables in kilograms?

The time variable for the pre-intervention represents a simple linear trend and is
equal to the number of months since entry into the study. The underlying trend is
assumed to be maintained throughout the study period but could be modified by the
intervention and the stimulus. The value of the coefficients were determined by
regression analyses on cash turnover, volume of sales and proportion of sales
relative to total sales as outlined in column 1 of the Table shown in Box 4. The
values of each of the time variables and the stimulus variable are made explicit in the
table in the Appendix.

In addition to these technical questions, another issue we are grappling with is

whether the switch-on dates for the stores properly correspond to the actual switch-
on dates of income management. A careful examination of table 1 in the paper
suggests some discrepancies in this respect. Could you please advise the location
of the stores and the corresponding switch-on dates used in the analysis?

We believe that the Arnhem Land Progress Aboriginal Corporation has provided
Boronia Halstead (Section Manager, Food Security Policy Intergovernmental and
Policy Branch) with the names of the study communities. We are bound by ethics on
what information we can disclose. Disclosure of data or names of community would
require approval from the HREC and a letter of agreement from all collaborating
parties.

Boronia has confirmed the income management switch-on dates for four of the

3
communities with Julie Brimblecombe. These dates were provided by FaHCSIA
when Menzies conducted an evaluation of the ALPA FOODcard. This report was
provided to FaHCSIA in 2008 and provided some preliminary results regarding
income management and store sales. The switch-on dates for the other six study
communities shown in Box 1 are the dates when income management went live in
each of the study communities according to ALPA. It is important to note that for
community 1 for example, although the switch-on date was the 25th March 2008, the
income management starting date on which the analysis is based is 1st April as the
unit of analysis was monthly store sales data.

4
Appendix:

An example of the relationship between the time and stimulus variables for one
community

t ti tp stimulus notes
1 0 0 0 entry into study
2 0 0 0
3 0 0 0
4 0 0 0
5 0 0 0
6 0 0 0
7 0 0 0
8 0 0 0
9 0 0 0
10 0 0 0
11 0 0 0
12 0 0 0
13 0 0 0
14 0 0 0
15 0 0 0
16 0 0 0
17 0 0 0
18 0 0 0
start of intervention (this date differs depending on the income
19 1 0 0 management go-live date for each of the communities
20 2 0 0
21 3 0 0
22 4 0 0
23 5 0 0
24 6 0 0
25 0 1 0 period 6 months after intervention
26 0 2 1
27 0 3 1 stimulus period
28 0 4 1
29 0 5 0
30 0 6 0
31 0 7 0
32 0 8 0
33 0 9 0
34 0 10 0
35 0 11 0
36 0 12 0