Evaluation 2009 Banner

Return to search form  

Contact emails are provided for one-to-one contact only and may not be used for mass emailing or group solicitations.

Session Title: Needs Assessment With Technology: Surveys, Contextual Factors, and the Community
Multipaper Session 293 to be held in Panzacola Section G2 on Thursday, Nov 12, 1:40 PM to 3:10 PM
Sponsored by the Integrating Technology Into Evaluation
Chair(s):
Wendy DuBow,  University of Colorado at Boulder, wendy.dubow@colorado.edu
Discussant(s):
Margaret Lubke,  Utah State University, mlubke@ksar.usu.edu
Survey Tools: An Embarrassment of Riches
Presenter(s):
Jamie Drakos, Cadmus Group Inc, jamie.drakos@cadmusgroup.com
Bonnie Watson, Cadmus Group Inc, bonnie.watson@cadmusgroup.com
Jessica Aiona, Cadmus Group Inc, jessica.aiona@cadmusgroup.com
Abstract: This paper reviews the efforts an evaluation firm, The Cadmus Group, used in selecting a software package for in-house surveys. Surveys are an important component of most evaluations, and a quality survey software tool is key for successful data collection and analysis. Performing surveys in-house rather than using a survey firm can offer benefits such as: reducing costs, capitalizing on internal staff's technical knowledge, and controlling survey administration. However, selecting a software tool from the sizable number of programs available can be daunting. Faced with this challenge, the authors reviewed several survey tools, paying particular attention to: ease of use, the number of full-access users the tool supported, capabilities to deliver surveys on-line, simultaneous respondent access, a CATI system, data storage on a secure server, data extraction, and compatibility with analysis software. This paper describes several software packages and presents pros and cons for the survey tool chosen.
Contextual Factors in Web-based Surveying: A Longitudinal Investigation of Rates of Response Patterns Within the Response Window
Presenter(s):
Sheryl A Hodge, Kansas State University, shodge@ksu.edu
Christa A Smith, Kansas State University, christas@ksu.edu
Valerie K York, Kansas State University, vyork@ksu.edu
Abstract: Dillman's (2007) update on the tailored design method as it relates to Internet surveys suggests continued use of mixed-mode techniques for minimizing sampling bias and achieving optimal population representation. Moreover, as with mailed surveys, Ritter and Sue (2007) underscore the natural dependency of Internet survey response rates on participant's investment, incentives, reminders, and specific subject attributes. Clearly, a representative sample of the population is imperative for replication and external validity, and respondent characteristic analyses are equally important in bolstering internal validity. Although the tailored design method was foundational for sound Internet surveying, it is vital that further idiosyncrasies inherent to Web-based surveying continue to be investigated. Inasmuch, this study will extend earlier research on rates of Web-based survey response within the data collection window (Hodge & Smith, 2006; Smith, Hodge, & McGee, 2007) by presenting longitudinal support for response patterns, accompanying demographic characteristics, and within-subjects analyses of increased incentive effects.
Evaluating Community in Online Settings: Frameworks, Evidence and Techniques
Presenter(s):
Vanessa Dennen, Florida State University, vdennen@fsu.edu
Abstract: The term 'community' is used frequently to refer to users of particular interactive online tools, but how do we know that true community exists? This presentation addresses this issue by covering two topics. First is a discussion of frameworks and criteria for identifying online community, with a focus on surface (e.g. links, tags) and deep (e.g. comments, norms) evidence that can be found in online settings. Second is a discussion of data collection and analysis techniques, including social network analysis and naturalistic methods, that may be used when evaluating the presence of virtual community. An evaluation of a blog-based community will be used to provide concrete examples throughout.

 Return to Evaluation 2009

Add to Custom Program