History: aw2l 3.1 Results

Results

There where a total of 229 surveys and 223 auto-recording tables collected (divided in methodologies and subjects in Table 1 in the accompanying communication; De Pedro, 2006).

Among the items answered in the collected surveys, some results are shown specifically for the subject "Evaluation of Environmental Impact, course 2004-2005" (AIA 0405) in detail, from which the more extensive and balanced data-set was gathered (circa 30 students with Traditional methodology versus circa 30 with Wiki methodology).

Methodological issues:

Students with Wiki methodology did not use as much the more common ways of communication (phone, chat or instant messaging, e-mail) and of exchange of documents (floppies, USB-drives, ...), but they employed more the Wiki technology. Moreover, meetings in person were reduced in the groups of students that followed Wiki methodology.

Organisational issues

In general, first-cycle students showed less difficulties to hold meetings during the week than those of second cycle. This is totally justifiable due to the bigger dispersion in subjects of last-years students, in spite of the case that subject was compulsory or optional in the degree.
The difficulties of internal organisation inside groups were very heterogeneous, and they did not seem to be tied to the work methodology but more to the characteristics of each group of students.

Marks and time yield

In very big work groups (AIA, 15 students for group) there is a clear advantage in the yield of the time invested for the students that worked using Wiki methodology respect to the ones using Traditional, with more frequent occurrence of high marks, for the same or less time invested (Figure 1).



In smaller groups (2-4 pupils for group, generally), where it is not essential to use special methodologies of work in group, there are not evident differences, and the success in the use of the tool might be more related with other factors (described in the full communication of this presentation)

Student satisfaction

There are some differences in the perception of the satisfaction about the method between the pupils of first cycle and those of second cycle. On the one hand, students of first cycle compulsory subjects, if they can meet every day in class and that are more usual to the multimedia technologies (Functional Ecology), then they need more tutelage and direction of the work, and they do not seem to appreciate the advantages of the methodology so much. The students of optional subjects of the first cycle that can not meet every day in class (because most of them work besides studying, as it happens in SIMA "Health and Environment" from Nursery degree), they do appraise it indeed more positively than with the traditional method.

Students of second cycle feel satisfied and recognise the advantages of the Wiki methodology, especially regarding the subject AIA and EAUB. In some specific cases, however, they did not use the tool due to technological handicaps and because they were not used to this methodology of work.

"AIA 0405" specific case study
Students that used Wiki methodology showed a higher satisfaction degree for the tool than the students that didn't use it (Figura 2. QS 39 – AIA 0405), and they showed a little less satisfaction regarding the Internet connection (Figura 3. QS44 – AIA 0405).



Students that worked with Wiki thought the the methodology they used served them to increase their final quality in the work and document, and the ones who used traditional methodology didn't think that their work methodology helped them as much as their colleagues with Wiki methodology thought (Figure 4. Q49 – AIA – 0405).

Likewise, all students thought that Wiki methodology facilitated that they couild interact with the parts of the document led by other members of the work group, and that the other ones could interact with theirs. This can be deduced from that students with Wiki methodology had the feeling that it allowed them to improve their interaction (Figure 5. Q50 – AIA 0405), and the others declared that traditional methodology, with which they worked, didn't facilitate this expected (and pursued by lecturers as a goal from the subject objectives) interaction among students.

Time invested by items

It could be analysed only in four subjects (AIA-0304, AIA-0405, EAUB and SIMA). Some of the items were specific of each subject and their comparison has no real meaning in practise for all subjects, since the survey, common to all, contained questions about items that were not used at all in some subjects, but were in others.

Thus, we highlight (in bold) some items out of the 16 proposed (Annex 2), due to its interest for the global comparison:

• T1 = Chosing the topic
• T2 = Searching information
• T3 = Structuring the document information,
• T4 = Writing of the work,
• T5 = Including images in the document
• T6 = Process of giving homogeneity to style and content,
• T7 = Final markup to the document, in web page
• T8 = Final markup of the document, in paper,
• T9 = Preparing oral presentation
• T10 = Exchange of files and materials,
• T11 = visits-interviews with topic experts
• T12 = Journeys,
• T13 = Tutorship in person with teacher/s
• T14 = Tutorship by telematic means with teacher/s
• T15 = Meetings in person of the members of the group, and
• T16 = Telematic communication among the members of the group: phone, wiki, e-mail, chat, etc.

The time devoted to structuring the work (item T3) is not necessarily inferior using Wiki methodology respect to traditional methodology, even though there was an exception (AIA 0405).

The time devoted to writing the work (item T4) has been inferior with Wiki methodology. This is a positive result of support to the Wiki tool and methodology.

Another aspect that we wanted to explore was if the Wiki methodology favoured the process of giving homogeneity to the markup, style and content of the work (item T6). Students from AIA and SIMA subjects that worked with Wiki methodology dedicated less time to giving homogeneity than those with traditional methodology, except for the EAUB subject, having in all cases similar presentation quality. It remains unclear if these differences in results are only a consequence of the type of methodology or a consequence of the different interest and commitment of each working group in the matter of work.

In all experiences, Wiki methodology saved, in average for the whole group, about 50% of the time invested on final markup of the work (item T8), compared to students working with traditional methodology. But the teaching staff felt that the single students that made this markup did not have to invest less time through Wiki methodology than the ones with traditional methodology. Suitable data was not collected to verify this hypothesis.

In general, Wiki methodology enables a time saving in the exchange of files and materials (item T10) of about 50%.

Regarding the time dedicated to journeys (T12), even though it seemed that the Wiki methodology saved time in general, results were not conclusive since the time for journeys was tied to the specific characteristics of each work (there were works where longer journeys were required for interviewing experts, and on this matter, Wiki methodology was not able to bring savings).

The time devoted to meetings in person (T15) was in global reduced with Wiki methodology in a clear way, even though in a single case the results were inverted (to see EAUB).

"AIA 0405" specific case study

On the one hand, students that used Wiki methodology invested less time in (Figure 6):

• choosing the topic (item 1 or T1),
• structuring the document information (T3),
• homogenization of the single discourse of the document (T6),
• exchange of files and other material (T10), and
• meeting in person of the work group members (T15).

On the other hand, Wiki methodology meant a major inversion in time, respect to traditional methodology, in:

• including images in the document (T5, although there were very little students that included images in the document, in both work methodologies).
  This result was expected since we there was a missconfiguration of Tikiwiki in our specific hosting provider, which was solved long ago, but after the experiences were carrried with students in the framework of the project reported here (course 2004-2005). By then, students had to upload images not directly to the single page were they were going to use them (as explained in De Pedro 2006, which is probably less time consuming than including them in MS Word documents), but they had to upload them to "Image Galleries" feature, search them there, copy their link, and go back to the text document to include them there by manually adding some wiki code to reference that image in that position. This longer and more manual process obviously meant longer times to include images, and also some disagreement from students (and teachers), of course, which had to learn the process to include them in the wiki page already. Many students didn't include images in the wiki pages, but waited to have the final OOo Writer or MS Word document to include them there, at that final stage of the group work, just before printing it in paper.

Tutorship (T13, T14) is not commented here but in a next section (below).

Tutorship

Students who had worked with Wiki methodology invested more hours of tutorship in person in all cases with data ("AIA0405" and "SIMA"; both with the same teacher) than their colleagues with traditional methodology. They had to do some extra meetings inside class hours for showing them the basic functioning of the Wiki technology and solving doubts, since only between 5 % and 10 % of the those students had done the specific extra course (convalidable as "free choice" credits) on Wiki technology offered. Looking at data from these specific students who attended the extra course, the time they invested in tutorship in person was much lower than the average (data not shown).

Time dedicated to tutorship through telematic means (e-mail. forums, internal Tiki messaging, etc.) was much lower in students with Wiki methodology than in those who used Traditional methodology, for the most relevant case ("AIA 0405"), where students knew already e-mail, web browsing, searching information in computers, etc. Students from "SIMA" (just arrived to university, and most of them with much lower previous knowledge in computer science than students from "AIA 0405", which is a more difficult degree to roll in at the beggining, and students at third year at university), behaved differently, and in which the pupils that did not use the methodology Wiki do not make to consume a lot of time in tutorship on the achievement of his work.

"AIA 0405" specific case study
Those who have worked with Wiki showed to have invested more hours in tutorship or support of the professorship at level presencial that the pupils who worked with traditional methodology (Figure of the item 13 - AIA 0405), but not at telematic level, in which they invested less (Figure of the item 14 - AIA 0405).

The different tutorització presencial might assume to the biggest need to consult with the teacher that those that the Wiki is writing itself in or speaking for the electronic forums of the subject. Whereas the pupils with traditional methodology would speak already more in person between the different members of the group of work, and therefore, he would be necessary to go to consult less basic things to the professorship.

On the other hand, the pupils who worked with methodology Wiki would have needed minor time of tutorització for route telematic, since the Wiki already would allow to ask about them to comment or even to discuss concrete things of the work without having to be presencialment.



Did not the pupils who worked with wiki stay all been satisfied with the contributed support the professorship to realize the practices, whereas those who used the traditional method of work in team if that remained very satisfied (Figure Q23? AIA 0405). Of here it is deduced that, being the task of the professorship the same one with regard to the content of the subject and rules for the holding a permit but autonomous work of the students, the difference must be to cause specifically of the method of work based on Wiki. I joust that is to say that only 4 of 31 pupils who used the Wiki signed up previously the extraordinary curset of 10 hours (convalidable as credits of free election) on writing cabbage · laborativa with Wiki, receiving the rest, little more than two hours of formation in Wiki distributed along the first two meetings attend them with the pupils. This way, it would be of hoping that the pupils who worked with methodology Wiki should remain with the sensation that them it was necessary to have more support metodològic (for that the support on the content was the same for some and others) on the part of the professorship.



Consequently it seems importing that the pupils who have to be employed with methodology Wiki at his subject, could have made earlier a specific curset to get accustomed in a slow, gradual way and comfortable to the especificitats of the methodology of work Wiki to end and effect of extracting the maximum profit with the minimum of desorientació due to the methodology of work cabbage · laboratiu based on Wiki the day that they had to use at level of production in real situations of teams of work cabbage · laboratius.

Results

There where a total of 229 surveys and 223 self-recording tables collected (divided in methodologies and subjects in Table 1 in the accompanying communication; De Pedro, 2006).

Among the items answered in the collected surveys, some results are shown specifically for the subject "Evaluation of Environmental Impact, course 2004-2005" ("AIA 0405") in detail, from which the more extensive and balanced data-set was gathered (circa 30 students with Traditional methodology versus circa 30 with Wiki methodology).

Methodological issues:

Students with Wiki methodology did not use as much the more common ways of communication (phone, chat or instant messaging, e-mail) and of exchange of documents (floppies, USB-drives, ...), but they employed more the Wiki technology. Moreover, meetings in person were reduced in the groups of students that followed Wiki methodology.

Organizational issues

In general, first-cycle students showed less difficulties to hold meetings during the week than those of second cycle. This is totally justifiable due to the bigger dispersion in subjects of last-years students, in spite of the case that subject was compulsory or optional in the degree.
The difficulties of internal organization inside groups were very heterogeneous, and they did not seem to be tied to the work methodology but more to the characteristics of each group of students.

Marks and time yield

In very big work groups (AIA, 15 students for group) there is a clear advantage in the yield of the time invested for the students that worked using Wiki methodology respect to the ones using Traditional, with more frequent occurrence of high marks, for the same or less time invested (Figure 2).



In smaller groups (2-4 pupils for group, generally), where it is not essential to use special methodologies of work in group, there are not evident differences, and the success in the use of the tool might be more related with other factors:

• students were appropriately motivated by teachers
• students controlled, previously to their use in class for production, the functioning of the new tool that they were going to use. This would imply that all students would have joined necessarily some suitable specific guided formation on the Wiki methodology, and on the correct use of the Wiki technology
• students had difficulties to meet in person (all the members of the work group), with as much frequency that they felt it was necessary for them
• students didn't refine at the end the writing, markup, grammar and spell checking, removing duplicities of information, synthesizing again, if needed, etc.

Student satisfaction

There are some differences in the perception of the satisfaction about the method between the pupils of first cycle and those of second cycle. On the one hand, students of first cycle compulsory subjects, if they can meet every day in class and that are more usual to the multimedia technologies (Functional Ecology), then they need more tutelage and direction of the work, and they do not seem to appreciate the advantages of the methodology so much. The students of optional subjects of the first cycle that can not meet every day in class (because most of them work besides studying, as it happens in SIMA "Health and Environment" from Nursery degree), they do appraise it indeed more positively than with the traditional method.

Students of second cycle feel satisfied and recognize the advantages of the Wiki methodology, especially regarding the subject AIA and EAUB. In some specific cases, however, they did not use the tool due to technological handicaps and because they were not used to this methodology of work.

"AIA 0405" specific case study
Students that used Wiki methodology showed a higher satisfaction degree for the tool than the students that didn't use it but knew about it because their colleagues told them the details (Figure 3), and they showed a little less satisfaction regarding the Internet connection (Figure 4). This could imply that even if some colleagues tell a student about some advantages and satisfaction using Wiki methodology, they only become satisfied with it when they try it themselves and have a good experience with it (and not just due to comments from the teacher or their colleagues about it).

Figure 3. Satisfaction of students about Wiki methodology. Case of subject "AIA 0405

Figure 4. Satisfaction of students regarding the Internet connection. Case of subject "AIA 0405

Students that worked with Wiki thought the the methodology they used served them to increase their final quality in the work and document, and the ones who used traditional methodology didn't think that their work methodology helped them as much as their colleagues with Wiki methodology thought (Figure 5).

Likewise, all students thought that Wiki methodology facilitated that they could interact with the parts of the document led by other members of the work group, and that the other ones could interact with theirs. This can be deduced from that students with Wiki methodology had the feeling that it allowed them to improve their interaction (Figure 6), and the others declared that traditional methodology, with which they worked, didn't facilitate this expected (and pursued by lecturers as a goal from the subject objectives) interaction among students.

Figure 5. Perception from students that the methodology they used improved final quality of the work. Case of subject "AIA 0405"

Figure 6. Perception from students that the methodology they used improved interaction among the members of the work group. Case of subject "AIA 0405"

Time invested by items

It could be analyzed only in four subjects (AIA-0304, AIA-0405, EAUB and SIMA). Some of the items were specific of each subject and their comparison has no real meaning in practice for all subjects, since the survey, common to all, contained questions about items that were not used at all in some subjects, but were in others.

Thus, we highlight (in bold) some items out of the 16 proposed (Annex 2), due to its interest for the global comparison:

• T1 = Choosing the topic
• T2 = Searching information
• T3 = Structuring the document information,
• T4 = Writing of the work,
• T5 = Including images in the document
• T6 = Process of giving homogeneity to style and content,
• T7 = Final markup to the document, in web page
• T8 = Final markup of the document, in paper,
• T9 = Preparing oral presentation
• T10 = Exchange of files and materials,
• T11 = visits-interviews with topic experts
• T12 = Journeys,
• T13 = Tutorship in person with teacher/s
• T14 = Tutorship by telematic means with teacher/s
• T15 = Meetings in person of the members of the group, and
• T16 = Telematic communication among the members of the group: phone, wiki, e-mail, chat, etc.

The time devoted to structuring the work (item T3) is not necessarily inferior using Wiki methodology respect to traditional methodology, even though there was an exception (AIA 0405).

The time devoted to writing the work (item T4) has been inferior with Wiki methodology. This is a positive result of support to the Wiki tool and methodology.

Another aspect that we wanted to explore was if the Wiki methodology favored the process of giving homogeneity to the markup, style and content of the work (item T6). Students from AIA and SIMA subjects that worked with Wiki methodology dedicated less time to giving homogeneity than those with traditional methodology, except for the EAUB subject, having in all cases similar presentation quality. It remains unclear if these differences in results are only a consequence of the type of methodology or a consequence of the different interest and commitment of each working group in the matter of work.

In all experiences, Wiki methodology saved, in average for the whole group, about 50% of the time invested on final markup of the work (item T8), compared to students working with traditional methodology. But the teaching staff felt that the single students that made this markup did not have to invest less time through Wiki methodology than the ones with traditional methodology. Suitable data was not collected to verify this hypothesis.

In general, Wiki methodology enables a time saving in the exchange of files and materials (item T10) of about 50%.

Regarding the time dedicated to journeys (T12), even though it seemed that the Wiki methodology saved time in general, results were not conclusive since the time for journeys was tied to the specific characteristics of each work (there were works where longer journeys were required for interviewing experts, and on this matter, Wiki methodology was not able to bring savings).

The time devoted to meetings in person (T15) was in global reduced with Wiki methodology in a clear way, even though in a single case the results were inverted (EAUB).

"AIA 0405" specific case study

On the one hand, students that used Wiki methodology invested less time in (Figure 7):

• choosing the topic (item 1 or T1),
• structuring the document information (T3),
• homogenization of the single discourse of the document (T6),
• exchange of files and other material (T10), and
• meeting in person of the work group members (T15).

On the other hand, Wiki methodology meant a major inversion in time, respect to traditional methodology, in:

• including images in the document (T5, although there were very little students that included images in the document, in both work methodologies).
  This result was expected since we there was a misconfiguration of TikiWiki in our specific hosting provider, which was solved long ago, but after the experiences were carried with students in the framework of the project reported here (course 2004-2005). By then, students had to upload images not directly to the single page were they were going to use them (as explained in De Pedro 2006, which is probably less time consuming than including them in MS Word documents), but they had to upload them to "Image Galleries" feature, search them there, copy their link, and go back to the text document to include them there by manually adding some wiki code to reference that image in that position. This longer and more manual process obviously meant longer times to include images, and also some disagreement from students (and teachers), of course, which had to learn the process to include them in the wiki page already. Many students didn't include images in the wiki pages, but waited to have the final OOo Writer or MS Word document to include them there, at that final stage of the group work, just before printing it in paper.

Tutorship (T13, T14) is not commented here but in a next section (below).

Tutorship

Students who had worked with Wiki methodology invested more hours of tutorship in person in all cases with data ("AIA0405" and "SIMA"; both with the same teacher) than their colleagues with traditional methodology. They had to do some extra meetings inside class hours for showing them the basic functioning of the Wiki technology and solving doubts, since only between 5 % and 10 % of the those students had done the specific extra course (convertible as "free choice" credits) on Wiki technology offered. Looking at data from these specific students who attended the extra course, the time they invested in tutorship in person was much lower than the average (data not shown).

"AIA 0405" specific case study
Time dedicated to tutorship in person was higher in students with Wiki methodology (Figure 8), whereas it was the opposite when tutorship was requested through telematic means (e-mail. forums, internal Tiki messaging, etc.) was much lower in students with Wiki methodology than in those who used Traditional methodology, for the most relevant case ("AIA 0405"), where students knew already e-mail, web browsing, searching information in computers, etc (Figure 9).

The different tutorship in person might be referred to their biggest need to consult the teacher the topics what are bring written in the Wiki or discussed in the electronic forums of the subject. Whereas students from traditional methodology would already speak more in person among the different members of their work group, and therefore, they would need less to go to the teacher to consult basic things.

On the other hand, the pupils who worked with Wiki methodology would have needed less time of tutorship through telematic means, since the Wiki would already allow to ask, to comment or even to discuss them all about specific things of their work or document without having to meet in person.

Figure 8. Time invested in "in person" tutorship. Case of subject "AIA 0405"

Figure 9. Time invested in tutorship through telematic means. Case of subject "AIA 045"

Not all students who worked with Wiki methodology were satisfied with the contributed support by their teacher to do the work (in the frame of the assignment in the practical classes of AIA subject), whereas those who used the Traditional methodology they did remained very satisfied (Figure 10). It can be deduced from here that, being the tasks and intended support from the teacher the same towards all groups of students (regardless of the work methodology they used), the difference had to be specifically on the work methodology based on Wiki. It has to be stated, also, that only four out of 31 students who used the Wiki joined previously the extraordinary course of 10 hours (convertible as "free choice" credits) on collaborative writing using Wiki. The rest received little more than two hours of formation / training in Wiki usage, distributed along the first two meetings in person with students within the practical classes of Evaluation of Environmental Impact. This way, it would be expected as logical that students who worked with Wiki methodology should remain with the sensation that it would have been more necessary for them to have had increased methodological support (because the support about the specific content from AIA subject was them same for all groups of students).



Consequently, it becomes important that fact that, students who have to use Wiki methodology, could previously receive a specific course to get used to Wiki methodology (no only to Wiki technology) through a slow, gradual and comfortable way. This way, they will be able to extract the maximum profit out of their work with the minimum disorientation due to the collaborative work methodology based on Wiki, whenever they have to use it at production level in real situations (other subjects at university or in their professional jobs out of university later on).