[Social-sciences-list] Anthropology

Sat Mar 14 22:59:41 UTC 2015

Hello all,

To be clear, I did not question re-solicitation which is the right action to take in many circumstances. I questioned the justification for doing so because of an imbalance between the female and male respondent numbers.

There is no need for the numbers to be balanced and in some situations doing so is counter-productive. In fact, gender is something we do understand in most populations so well that balancing a sample to approach the population's make-up is far from necessary unless there are other reasons for doing so. One is that the actual number of respondents for one group is too small to be sure that the findings have any transferability. In the ongoing exchange between Nora and me off-list, we talked about the need to get the sample to have the qualities of interest not the quantities. To follow on Christine's point, getting the male group to match the female group should not be done on equivalent sample size but on other grounds that match the purpose of the work being done. If that includes numerical summaries, then the motivation might be to get similar standard errors for subgroups, which is a combination of variance and sample size so the greater the variance, the greater the subgroup being sampled should be. More commonly though, is the need to get a variety of respondents that can be demonstrated as representative of a wider population.

To touch on the additional points just made by Vincent, I would say that he and others should think carefully about the differences between surveys and experiments. I assume given what little I know of his plans, that to test his ideas will require planning an experiment not a survey. While his set of experimental units might be a non-random sample of some wider population, he will at least have the ability to randomise the participants into various treatment groups.

I do appreciate the fact that Arielle and Christine demonstrate an understanding of the limitations that result from convenience sampling in their messages. This is really worth applauding as it is in my experience actually not well understood in general. My off-list  discussion with Nora showed greater understanding than I would expect of someone who has not yet started a PhD.

Finally, you may ask about the basis for my concerns with respect to the blind population. I attended an international conference last year where I endured (mostly in silence) a number of presentations where blind people were used in poorly designed experiments and surveys with staggeringly small samples. One experiment had 8 individuals in each of two groups (surprising that they didn't find a significant result!) and another admitted after being questioned that there were only 12 respondents in total, because she was  reporting on the blind students studying at a single university. If that speaker had limited her presentation to the qualitative responses from her respondents and the breadth of response observed I would have felt less that she had wasted 20 minutes of my life. I still haven't worked out how the time taken for the respondents to complete a task could be described as "a little under 20 minutes, with a standard error of just over 10 minutes". Even if the speaker had meant a standard deviation, I'd be worried about the way she thought the numeric results had any value to the audience and the supposed value of her research to the blind of the world.

I am not sorry if people think my comments are a little disparaging; they are meant to be. Poor research practice is going on out there and all too often I see it involving blind people. When proper research is being conducted we need to applaud it, especially when it is done by one of our own. This list can be a great way to support the improvement of research being done "on" blind people by helping when it is being conducted "by" blind people. 

Cheers,
Jonathan

-----Original Message-----
From: Social-sciences-list [mailto:social-sciences-list-bounces at nfbnet.org] On Behalf Of Arielle Silverman via Social-sciences-list
Sent: Sunday, 15 March 2015 8:01 a.m.
To: Nora Goldberg; Blind Social Scientists List
Subject: Re: [Social-sciences-list] Anthropology

Hi all, I agree with Christine. I am a postdoctoral researcher with several years of stats training and research experience. Yet as a rehabilitation researcher, studying very specialized populations like the blind or people with spinal cord injuries, it is virtually impossible to obtain a truly random sample. We attempt to oversample certain groups (e.g. males, ethnic minorities, people from lower SES
backgrounds) for this reason. It is generally accepted in the field that convenience sampling is the best we can find. We acknowledge this as a limitation in our papers. So I don't see anything wrong with re-soliciting to certain subgroups of people as Christine described.
Best, Arielle

On 3/14/15, Nora Goldberg via Social-sciences-list <social-sciences-list at nfbnet.org> wrote:
> Hi Dr. Szostak, Dr. Godfrey, and all Social Science Listers,
>
> Thanks for reaching out and for your concern, Dr. Godfrey.  I am 
> completely aware that my survey, and thus my ethnography itself, is 
> biased.  The reason I mentioned that the mix of male to female is 1:4 
> is because I really want to get more male voices and male 
> perspectives.  Or, statistical power, as Dr. Szostak pointed out.  
> Another reason I made this distinction is because using my data I have to come up with a series of hypotheses.
> For example, "If a person is blind or visually impaired and is female, 
> then she will..."  I am also going to use age as a variable, and 
> highest level of education attained, etcetera.  But if there is only 1 
> male response for every 4 female response then I really shouldn't be 
> drawing any conclusions by gender.  And yes, the absolute number of 
> males is comparatively less than females.
>
> I could not have chosen this topic, and used a random sample, simply 
> because I do not know enough blind or visually impaired folks to be 
> able to acquire any sort of meaningful data.  I spoke with a number of 
> people, all blind or visually impaired, in person and on the phone 
> before I even considered surveying as my method of research.  It was 
> the President of the Pennsylvania chapter of the AFB suggested I 
> contact people using the Listserv.
>
> Students in my class have a whole section of our ethnography about the 
> limitations of our method of data collection, so all this will be 
> taken into consideration.  And yes, only hypotheses can be made about 
> the respondents and not the community at large, and no direct 
> conclusions can be drawn.  Only speculations based on the interpretation of the data.
>
> I am only an undergraduate student and this course is very low 
> stakes--it is merely an introductory course to ethnographic methods.  
> (I am in the Honors College, although this is not for an honors class 
> or project.)  I could have chosen any culture or community to observe 
> and I chose the blind and visually impaired community.  In fact, at 
> first it was not working out because I had lots of trouble finding a 
> cultural broker/informant, but I persisted because I really wanted to 
> pursue this topic.  Though meanwhile I could have jumped ship any 
> number of times.  If the ethnography was going to be any more robust 
> than it is, then I would have had more than 10 weeks to choose site, 
> learn how to effectively take field notes, choose a topic within the 
> site, operationalize the topic, create hypotheses, choose a
> method(s) of data collection, gather data, analyze the data and 
> complete the writeup.  As Dr. Szostak and others along the way have 
> pointed out, if this were for a graduate course I would have had to 
> gone about the project much differently.  Personally that discrepancy 
> bothers me, I don't know about anyone else.
>
> Thank you Dr. Godfrey and Dr. Szostak for your thought provoking 
> questions and comments.  For all the help that this Listserv is 
> providing me with, the least I can expect in return is to be given a 
> run for my money :-)
>
> Sincerely,
>
> Nora
>
>
> On Fri, Mar 13, 2015 at 8:17 PM, Szostak, Christine via 
> Social-sciences-list <social-sciences-list at nfbnet.org> wrote:
>
>>  Hi Nora and All,
>>
>>   Though I fully agree with the concerns mentioned below, since, if 
>> memory serves, this is for an undergrad honors project, I would not 
>> worry too much about these issues. If you are going on to graduate 
>> training, you will definitely want to take the below to heart. 
>> However, there is nothing wrong with soliciting further for the parts 
>> of your sample that are under-represented. In my research, I study 
>> both native and non-native speakers of American English and find 
>> myself often having to re-solicit to one group or the other so that I 
>> can properly balance my samples and match them as well as possible.
>>
>>
>>
>>   This is unfortunately, for many of us, one of those things that you 
>> need to do to make sure  that your comparison groups are equally 
>> represented (e.g., if you carry out stats with groups of very 
>> different sizes, even summary stats like means) can be highly 
>> misrepresentative. Thus, I applaud your efforts to try to re-solicit 
>> to ensure your sample sizes are strong.
>>
>>
>>
>>   When you go to write this up, what you will need to do, to deal 
>> with the below concerns, is state that your sample was a sample of 
>> convenience.
>> For
>> many of us in the social sciences this is what we constantly face and 
>> have to accept  given our means…
>>
>>   In the future, one way to best describe this when talking with 
>> other researchers, would be to use phrases like statistical power  or 
>> to ensure that the data are more stable…
>>
>> Best of luck with the project!
>>
>> Chris
>>
>>
>>
>> Dr. Christine M. Szostak
>>
>> Assistant Professor of Psychology
>>
>> Department of Social Sciences
>>
>> Shorter University
>>
>> Rome, Georgia
>>
>> szostak.1 at osu.edu
>>
>> cszostak at shorter.edu
>>
>>
>>
>> *From:* Social-sciences-list [mailto:
>> social-sciences-list-bounces at nfbnet.org] *On Behalf Of *Godfrey, 
>> Jonathan via Social-sciences-list
>> *Sent:* Friday, March 13, 2015 8:01 PM
>> *To:* Nora Goldberg; Blind Social Scientists List
>> *Subject:* Re: [Social-sciences-list] Anthropology
>>
>>
>>
>> Hi there,
>>
>>
>>
>> I’m a little concerned that you care about the mix of male to female 
>> participants when you don’t even have a random sampling scheme. Your 
>> ability to infer anything for a  population is limited anyway for so 
>> many reasons.
>>
>>
>>
>> If you had suggested that your absolute number of males is less than 
>> what you want I would still be concerned but less so.
>>
>>
>>
>> I know that many surveys are done using convenience sampling of some 
>> kind but I seriously hope your course staff are explaining how you 
>> can only make comments about your sample respondents and at most 
>> suggestive statements about the population the sample may have come 
>> from. In simple terms, your sample is biased and aside from a lack of 
>> gender balance, it will be very hard to know how it is biased.
>>
>>
>>
>> Sorry if I sound a little harsh; poor sampling is a hobby horse of 
>> mine at present. I’m seeing too many postgraduate students who have 
>> not understood the limitations of their research findings because 
>> they do not have a random sample. They are not limited to social 
>> science students either by the way. Many students we see come to our 
>> statistical consulting service are looking at business-oriented 
>> research questions.
>>
>>
>>
>> Jonathan
>>
>>
>>
>>
>>
>> *From:* Social-sciences-list [
>> mailto:social-sciences-list-bounces at nfbnet.org
>> <social-sciences-list-bounces at nfbnet.org>] *On Behalf Of *Nora 
>> Goldberg via Social-sciences-list
>> *Sent:* Saturday, 14 March 2015 12:49 p.m.
>> *To:* social-sciences-list at nfbnet.org
>> *Subject:* [Social-sciences-list] Anthropology
>>
>>
>>
>> Hi Social Science Leisters,
>>
>>
>>
>> I wanted to let you know that tomorrow will be the last day to take 
>> my survey that I have been circulating for my Anthropology class.  
>> For your convenience the link is below.  If you have any questions 
>> about the survey or the course that it is for please feel free to 
>> email me.  Roughly 3/4 of respondents so far have been female, so 
>> gentleman, please, help me out!
>> I
>> want the data to paint the most balanced sketch possible.  And also 
>> feel free to share the survey link, either personally or via social 
>> media, with any friends and acquaintances who are blind or visually 
>> impaired.  The project has been very insightful, and I have really 
>> appreciated everyone's answers thus far.
>>
>>
>>
>> The survey will be found here: http://goo.gl/UBHm7x
>>
>>
>>
>> Thank you so much!
>>
>>
>>
>> Sincerely,
>>
>>
>>
>> Nora E. Goldberg
>>
>> Urban Sustainability and Community Place making
>>
>> Pennon Honours College
>>
>> Drexel University, 2016
>>
>> _______________________________________________
>> Social-sciences-list mailing list
>> Social-sciences-list at nfbnet.org
>> http://nfbnet.org/mailman/listinfo/social-sciences-list_nfbnet.org
>> To unsubscribe, change your list options or get your account info for
>> Social-sciences-list:
>>
>> http://nfbnet.org/mailman/options/social-sciences-list_nfbnet.org/nor
>> a.goldberg6%40gmail.com
>>
>
>
>
> --
> Nora E. Goldberg
> Environmental Policy, Economics & Public Health Pennoni Honors College 
> Drexel University, 2016
>

_______________________________________________
Social-sciences-list mailing list
Social-sciences-list at nfbnet.org
http://nfbnet.org/mailman/listinfo/social-sciences-list_nfbnet.org
To unsubscribe, change your list options or get your account info for Social-sciences-list:
http://nfbnet.org/mailman/options/social-sciences-list_nfbnet.org/a.j.godfrey%40massey.ac.nz