Research Methods in Electrical Engineering Professor David Thiel

Research Methods in Electrical Engineering Professor David Thiel Centre for Wireless Monitoring and Applications Griffith University, Brisbane Australia 1

Purpose • To make attendees familiar with the process of rigorous research in an academic environment. • To encourage attendees to critically evaluate research papers they read. • To outline the processes required to undertake a research project. 2

Topics 1. Overview of the Research Process 2. Literature Search 3. Report Writing, Data Collection & Presentation 4. Statistical Analysis of Data and Sampling 5. Making a Presentation 6. Survey Research Methods 7. Review 3

Topic 1 Overview of the Research Process 4

What is Research? • Discovery of new things that have been independently verified by other professionals. • Something new to humanity (not just new to you or your group). 5

Good & Bad Research Examples • Case 1 A high school research paper • Case 2 A good idea • Case 3 Tested outcomes for a new idea 6

The Scientific Method Prior knowledge An idea The Outcome is Recognised Submit as a Major contribution Report, Thesis, to the field Journal Discovery Independent verification: Independent experiment, literature, verification: literature, numerical model, analytical model, etc or Conference Paper Assessors 7

The Research Community • • All use the same scientific method. All follow the same ethical principles. All use the same language and terms. All provide information to the world-wide community reported in a full and open manner. • All acknowledge the previous work of others. 8

Publications and Referencing • The archival literature (must be printed somewhere and unalterable). • Must be reviewed by independent professionals before publication. • Must be complete so others can reproduce the results. • These three form the basic validity test! 9

Types of Publication • Scientific papers (refereed journal and conference papers) • Trade articles • Newspaper articles • Infomercials • Advertisements You must only rely on refereed papers in accredited journals and conferences. 10

How can you tell? • • Length of title References (and their quality) Author’s name and affiliation Evidence that the paper has been reviewed and revised. • Date of submission & date of publication. • The paper includes a review of previously published work. • Conclusion contains a critical reflection on the contents of the article. 11

Activity • Use http: //scholar. google. co. id/ and enter the key words from the paper you have. • Did you find it? • What else did you find? 12

“Next step” research • Incremental advance compared to paradigm shift. • Lateral translation research. 13

Topic 2 Literature Search 14

Literature Review • Who has done what and how? • What is their plan for “further work”? • Have they reported more recent work in a conference? • What opportunities are available for confirming the results of others and expanding their results and conclusions? 15

Key Words • Essential for searching the literature. • Must be both general and specific. 16

Publication delays • Check your paper and see the submission date and the publication date. • This delay mean that this team has moved forward with their research. Following their suggestions for further work might have you gazumped. • Conferences often have a 6 month delay between abstract submission and the conference presentation. 17

Planning for an outcome • What is convincing “proof”? • What is the evidence you will provide? – Data – Sampling techniques – Accuracy. • Who is interested in this research? • Where will you release (publish/present) your research results? 18

Anticipating problems • Team planning meetings – Circulate outcomes immediately following the meeting – Action items • Equipment calibration • Reliable power • Preventing Data loss 19

Publication of Data • Internal report? • Choosing a conference • Choosing a journal 20

Journal rankings • Impact factor • Half life • Citations (Google, ISI Thomson Web of Knowledge, Scopus, etc) http: //scholar. google. co. id/ • Weaknesses of the ranking systems • H index – The number of papers that have more than that number of citations fpr person. 21

Research Planning • Concurrent Engineering – Assembling the equipment – Arranging access to the site – Writing the paper draft – Choosing the journal • Concurrent Research 22

Using the right language • Definition of terms (standards, standard usage, standard methods of analysis). • Standard Measurement Procedures • Standard values (eg copper conductivity) • Spelling (US English or UK English? ), Lexicon and naming conventions. • Key words in publications • This is vital for accurate electronic searching of indexes. 23

Searching the Web • Google scholar http: //scholar. google. co. id/ • Journals and publisher’s indexes – IEEE Xplore digital library http: //ieeexplore. ieee. org/Xplore/dynhome. jsp – Elsevier http: //www. elsevier. com/wps/find/journal_brow se. cws_home – and many more. 24

IP Searching • Patents http: //www. uspto. gov/ http: //www. wipo. int/pctdb/en/searchadv. jsp • PCT Applications http: //www. wipo. int/pctdb/en/ • Country Based Searching http: //www. wipo. int/ipdl/en/resources/links. jsp 25

Activity • Find some scientific terms in your paper, and check the definition. (Why not wikipedia? ) • Key word searches, key word selection. • Definition of terms. 26

Topic 3 Report Writing 27

The title • 10 -15 words is most common. • Must be sufficiently specific. 28

The Abstract – an example • High speed electronic beam switching is a desirable feature of smart antennas. 29

The Abstract – an example • High speed electronic beam switching is a desirable feature of smart antennas. Most smart antennas are too large for most applications and require significant power during normal operations. 30

The Abstract – an example • High speed electronic beam switching is a desirable feature of smart antennas. Most smart antennas are too large for most applications and require significant power during normal operations. A thirteen element switched parasitic antenna was optimised for gain, speed and beam coverage. 31

The Abstract – an example • High speed electronic beam switching is a desirable feature of smart antennas. Most smart antennas are too large for most applications and require significant power during normal operations. A thirteen element switched parasitic antenna was optimised for gain, speed and beam coverage. Antenna characteristics were determined at 1. 8 GHz by finite element modelling and measurements on a prototype. 32

The Abstract – an example • High speed electronic beam switching is a desirable feature of smart antennas. Most smart antennas are too large for most applications and require significant power during normal operations. A thirteen element switched parasitic antenna was optimised for gain, speed and beam coverage. Antenna characteristics were determined at 1. 8 GHz by finite element modelling and measurements on a prototype. The antenna had a gain of +9. 8 d. Bi, a footprint of less than one half wavelength squared and was switched ion less than 100 ms. 33

The Abstract – an example • High speed electronic beam switching is a desirable feature of smart antennas. Most smart antennas are too large for most applications and require significant power during normal operations. A thirteen element switched parasitic antenna was optimised for gain, speed and beam coverage. Antenna characteristics were determined at 1. 8 GHz by finite element modelling and measurements on a prototype. The antenna had a gain of +9. 8 d. Bi, a footprint of less than one half wavelength squared and was switched ion less than 100 ms. This is a better performance compared to previous 34 antennas.

The Abstract – a general guide • 2 sentences on the wider field – context and significance. • 2 sentences on the research method • 2 sentences on the results and conclusions. 35

Scientific writing style Do’s and Don’ts • Past tense • Third person • Usually timing of events is not included unless it is essential to data collection. • Sections and subsections (one level? two level? three level? ). • Quotes from other authors – not common! 36

Creating equations • There are standard symbols for quantities (eg V=IR). • There are standard forms for scalar symbols (often lower case, italics, not-bold) and vector symbols (upper-case, bold). • The symbols must be the same font on every occasion used in the equations and in the main text. • All symbols must be defined. • MS Equation editor allows for equation creation. • There are standard upper-case and lower-case 37 type settings.

Data Collection & Presentation 38

Types of Data • Quantitative data (numerical) – Integers (eg animal counts, packets received, bit error rate) – Non-integers (eg analog sensor output) • Qualitative data (descriptive words) • Binary data (yes/no, success/failure, present/absent etc) • Scalar information (1 D, 2 D, 3 D, n. D) • Vector information (1 D, 2 D, 3 D, n. D) 39

Quantitative Data • Kelvin’s First Law of Measurement: “The measurement must not alter the event being measured”. – Microwave current measurements? – The impedance of an antenna? 40

Data Presentation • Plots (2 D and 3 D), histograms, pie charts, tables of numbers. • Printed papers usually black and white (lines distinguished by dots, dashes, ellipse, legend etc) • Colour in power point slides and web publishing. • For comparison plot more than one data set on the same graph using the same scale. • Images and flow charts. • Interpolation and extrapolation. • Curve fitting (covered in later lectures) • Contour plots. 41

Plotting and analysis tools • MS EXCEL (Chart Wizard - 4 steps) demonstration • Matlab (plot, subplot, contour, quiver, etc) 42

Graphing Guidelines • Always plot discrete points clearly. • Do not join points unless you have a continuous mathematical function. • To compare data plot several lines on the same axes. • Consider including error bars on all points 43

X X 44

Matlab scalar 2 D plots contourf surf image mesh 45

Matlab vector 2 D plots quiver North-south (metres) East-west (metres) 46

Qualitative Data • This can be a challenge as everyone will use a different description. • One approach is to convert qualitative data to quantitative data (eg rate from very bad to very good on a score of 1 to 10). 47

Decision Matrix Vehicle Cost Size Warranty Delivery time Comfort Total Score Mazda 3 6 8 7 8 8 37 Mazda 2 8 6 7 7 6 34 Ford Focus 6 7 7 8 7 35 Honda 6 6 5 28 Toyota Camry 4 8 6 7 8 33 VW 2 6 5 3 7 23 48

Decision Matrix - Histogram Score Survey Questions 49

Data Collection • Asking the right questions without leading the person (survey instruments questionaires). • Use redundant questions that always need a positive response (discussed in a later lecture). • Survey results (Is 35% return good enough? ). 50

Flow Charts (MS Word) Initiate equipment Yes/No? Stop process 51

Activity • Plotting analysis using MS e. Xcel. • Flow chart using MS word. 52

Topic 5 Statistical Analysis and Sampling 53

Normal Distribution From: http: //mathbits. com/Math. Bits/TISection/Statistics 2/normaldistribution. htm 54

Experimental error? • How does this compare with your results? • Is your result significant statistically? 55

Linear correlation • Need to fit a line to your data? Quote the linear correlation coefficient (linear regression) 56

Sampling • Population – every possible candidate. • Sample population – a small number of candidates selected from the population. • It is impossible to know from an examination of your sample alone, if your sample is representative of the whole population. 57

Examples: • In Australia the total population over 18 years votes in an election. Before the election, the press like to take a small sample the population to estimate the likely outcome of an election. This is called “polling”. They hope that the sample is representative of the entire population. How do they select a representative sample for a telephone poll? • • • – – Post code? Telephone book? In the street or shopping centre? etc 58

All samples may be biased • Why? – Age? – Shyness/openness? – Work times (shift workers)? – etc 59

Example • 6 people live in a single house • We want to randomly select 2 to get an idea of the use of mobile phones in the house. • To do this we could: visit at 10 am on a week day. visit at 3 pm on a week day. telephone at 8 pm on a week day. visit on Saturday morning at 10 am. Visit on Sunday afternoon at 3 pm. – etc 60

We ask the question: • How do you rate your use of a mobile phone on a scale of 1 to 10? – 10 means very continuously (>20 hours per week) – 1 means never (<30 minutes per week) 61

We have the following opinions • Mary stays at home, goes shopping and drives children to school at 8 am and pick up at 3 pm. • Fred drives to work for night shift. Leaves at 7 pm and comes home at 6 am. • Asif is a 9 am – 5 pm office worker who rides the train. • Sri is a part time sales person drives around the city from 10 am to 2 pm. • Chen cycles to University 9 am and back at 3 pm. • Rocco is retired and stays in the house all day. 4 2 5 8 7 1 Average value is 4. 5/10 62

How many possibilities? • If we select 2 people from the total population of n people we have P combinations where • ! indicates factorial where 5! = 5 x 4 x 3 x 2 x 1. • For a population of 6 we have 15 possibilities. 63

There are 15 different combinations • Lowest result from a sample of two people would be Rocco and Fred (2 and 1) – Mean is 1. 5/10. • Highest sample of two would be Sri and Chen (7 and 8) – Mean is 7. 5/10. • 5 combinations lie between 4 and 5 • 11 combinations lie between 3 and 6 • 13 combinations lie between 2 and 7 • 15 combinations lie between 1 and 8 64

Compromise required • The greater the need for a very accurate result, the smaller the chance of fulfilling this, even with the best method of approach. 65

Sampling Strategies • Clustered Sampling: Select a sample from only those parts of the population which are relevant; eg chose only those people who use the road at peak hour. • Stratified Sampling: Select a sample proportionally to those who are likely to use the road at peak hour and those that don’t. (4/6 use at peak hour and 2/6 don’t, so use a sample of 3, two who travel at peak hour and one that does not) • Destructive Sampling: If the sample is destroyed by sampling (i. e. their mind is changed), then clearly you should not sample all people. 66

Chassis strength testing • A production line of note book computers produces 2000 units per day. • The company is required to strength-test to failure 15 samples every day. • How do we select those samples? 67

The Monte-Carlo Method • A random sampling technique to define the effect of a large number of parameters on an outcome. (Usually between 0. 1% and 1% of total population). • Usually applied to complex systems described by mathematics. • One randomly selects the parameters and calculates the outcome. • Used in optimisation. 68

Random Sampling • How can I choose a team of 6 people randomly from this class? – Family name? – Student number? – Seating location in the class? – Every third person? • Every person must have an equal probability of being chosen. 69

Random Numbers 1 6 11 16 21 26 31 36 41 46 51 56 61 66 71 76 81 86 91 96 0. 5175 0. 3234 0. 4670 0. 3208 0. 0936 0. 8704 0. 4451 0. 1478 0. 6520 0. 9420 0. 8824 0. 6609 0. 2039 0. 1703 0. 0161 0. 1654 0. 1091 0. 3612 0. 5611 0. 9638 0. 2455 0. 0239 0. 0300 0. 8862 0. 8864 0. 9132 0. 5474 0. 1726 0. 4870 0. 8144 0. 9366 0. 5831 0. 5489 0. 4718 0. 7533 0. 3323 0. 1725 0. 5130 0. 3804 0. 8282 0. 9670 0. 0048 0. 3014 0. 4546 0. 8905 0. 8435 0. 2504 0. 7339 0. 8396 0. 4230 0. 7085 0. 4059 0. 5263 0. 5256 0. 0915 0. 4037 0. 7821 0. 2648 0. 3079 0. 1850 0. 7566 0. 6207 0. 6453 0. 3273 0. 1542 0. 1844 0. 4552 0. 5332 0. 1624 0. 9258 0. 4091 0. 0312 0. 1673 0. 5651 0. 9854 0. 1403 0. 3336 0. 3091 0. 3543 0. 1629 0. 3222 0. 3796 0. 6414 0. 6023 0. 0377 0. 3351 0. 0782 0. 5440 0. 4911 0. 2879 0. 2527 0. 4393 0. 6586 0. 3256 0. 0017 0. 9727 0. 1009 0. 3184 0. 9555 0. 3493 Excel function =rand() 70

Sample Rate • Number of samples per second. • In a digital recording sensor system this might be obvious initially, but there may be “overheads” when you need time to send and/or store data. • In an analog system this is regulated by the filter response (eg mechanical needle, DMM update speed, noise reduction filter). • Over-sampling and under-sampling. • Nyquist sampling (twice the maximum frequency of interest). 71

Topic 5 Making a Presentation 72

Preparing a Power Point Presentation • Maximum number of slides – one per minute! • Optimal number of slides – one per 2 minutes • Use slides as a reminder of what you will say. • During your presentation, do not read what is on the slides. • 100 words maximum on each slide. 73

Preparing a Power Point Presentation • Font size? (large!) • Graphs? (large!) • Colours? (clearly distinguishable, high contrast, minimal background colour – not dark) • Movies? (check on the presentation computer before your talk – usually they don’t work!) • Pictures? (not too dark) • Lighting? (Keep the room lights up so you can see the audience) 74

Images • You MUST acknowledge the source of image if it is not yours including – MS word image library (in this presentation) – Pictures taken from web sites – Pictures taken from colleagues – Graphs taken from papers etc 75

Organisation: 10 minute talk • • • Title slide (Name and affiliation) 1 Outline slide (Major sections) 1 Introduction (Wider research context) 1 Main text (method, apparatus, results) 4 -6 Conclusions 1 References 1 76

Nervous? • Hints for overcoming nervousness: • Memorise the first 2 -3 sentences (opening sentences). • Make sure you have key words on your power point to trigger your memory. • Do not start speaking until the title slide is visible to the audience. 77

Being Polite! Before you speak • Introduce yourself to the session chair before the session starts. • Load your presentation before the session starts. • Wait for the chair to introduce you before you speak. • Switch off your mobile telephone. 78

Being Polite! During your talk • Thank the chairperson for the introduction. • Speak clearly • Pretend you are talking to the back row of seats in the room (project your voice). • Acknowledge your co-authors in Slide 1. • Rigidly stick to the allocated presentation time. • Clearly indicate the presentation is finished by a slide and say “thank you” to the audience. • Do not invite questions from the audience. (This is the role of the chair person) 79

Being Polite! After your talk • Go quickly back to your seat. • Do not discuss your paper with others during the next talk. • If necessary, leave the room (politely – do not slam the door). • Once the session is complete, thank the chair person. 80

Why References? • For scientific rigour. • In case someone in the audience has made a major contribution to the field. • So the audience can follow up on your previous publications. 81

Topic 6 Survey Research Methods 82

• This is about how to prepare and analyse a survey (questionaire) 83

“Sick building” Survey • The research question: • Do you think that working in this building is making you feel sick? 84

Designing a Survey • Role of the researcher – Develop the research plan – Design the survey instrument – Select the sample population – Issue/distribute the survey – Prompt the sample population for responses – Analyse the data – Generate conclusions 85

Who are the stake-holders • Selecting the sample population – Who are the stake-holders? – What’s in it for them? (No interest can mean no completion) • Random selection from a large population • Inclusion – – Those that are keen to participate will respond – Are they a biased sample? • Exclusion – Will people be offended if they are not asked to respond? 86

Who are the stake-holders • You must be able to defend your sample population selection 87

Anonymous Responses • Arguments for “yes” – Anonymous – Sample population might be less influenced by who is asking the questions – Respondents might be less concerned about others learning of their opinions • Arguments for “no” – Non-anonymous – Who will you send the results to? – Who will you send the reward (chocolates) to? – How do you know who to follow up about returning the survey? 88

Confidentiality • You need to ensure that confidentiality is assured before the survey is sent out. • Consider using an independent third party to administer the survey. • I have been asked to complete a survey which asked for sufficient personal information to be identified uniquely. • How will you report “free” comments? 89

Feedback • It is assumed that your sample population (and the full population) will want access to the results. • You must explain how will this be done at the beginning of the survey. 90

Sample Time lines • Week 1: Pre-survey letter of introduction (Wider research context and brief research plan) • Week 2: Survey send out • Week 3: Mid-survey reminder letter • Week 4: Last minute final reminder • Week 6: Post-survey analysis report completed 91

Rating system – 5 point scale • • • Strongly disagree Disagree Neutral Agree Strongly agree 1 2 3 4 5 • Neutral allows respondents to “sit on the fence” 92

Rating system – 4 point scale • • Strongly disagree Disagree Agree Strongly agree 1 2 3 4 • This forces respondents to show positive or negative attitudes. 93

Topics for “Sick building” survey • • General personal well being Lighting Ventilation Noise and vibration Odour Electromagnetic radiation Security Demographics of respondents 94

Hints for writing questions • Keep is very simple – avoid jargon • Use one concept per question – avoid multiple concepts • Keep wording positive – avoid negative words and phrases, double negatives • The first question should be the “over-all question” – Never place a controversial question at the beginning. • Place demographics questions at the end – Demographics at the beginning can raise suspicions. • Keep related questions together – Difficult for the respondent to remain coherent • Use three questions per topic – Do not over question, don’t waste people’s time. 95

Statement wording • I don’t feel well most of the time (negative wording). • I enjoy good health. • I am satisfied with the ventilation and the lighting environment (double-barrelled question). • I am satisfied with the ventilation. • I am satisfied with the lighting. • The University does not do a bad job of keeping us informed about work place health and safety issues. (double negative) • The University does a good job of keeping us informed about work place health and safety issues. • Many students feel ill as soon as they walk into the building. (projecting the feelings of others). • Students enjoy working in this building. 96

Judgemental versus Observational • This work environment is just as good as other places where I have worked. • I am happy with this work environment. • The University listens and acts on student and staff concerns about the building environment. • I am satisfied with the University’s response to student concerns about the building environment. 97

Judgemental versus Observational • This work environment is just as good as other places where I have worked. • I am happy with this work environment. • What if you asked both statements to be rated? • The conclusions would be different 98

Reverse scoring • Q 10: I am not happy with this work environment. (1 – 5) • Q 35: I am happy with this work environment. (1 – 5) • You would need to reverse score Q 10 for proper statistics. • The dangers include: – Donkey vote gives confusion (What do you do if you get 5 for both? ) – Was the question misread? – Was the respondent annoyed? 99

Sample Open ended questions and comments • Please identify at least three things that cause you concern in this work environment. • Please identify at least three things that you like about this work environment. 100

Reporting • Calculate averages and statistics for each theme. • Construct a Histogram and report the mean value • E. g. 80% rated the noise environment neutral or better. • Or: 20% indicated that the noise environment was not good. • Report selective quotes on open questions. 101

Missing Data • Did the respondent simply forget one question? • Maybe the question was not relevant to that person? • Was the question too personal? • Was the question confusing? Could it have been scored as a 1 for one interpretation and a 5 using another interpretation. 102

Accuracy and Reliability • On a 5 point scale there are 5 possible answers. • Your mean value for the sample population can be expressed to several decimal places. • How many places are significant? • Return to Normal Distribution statistics based on z score. 103

References • Connolly, P. M. & Connolly, K. G. , 2004, Employee opinion questionaires, Pfeiffer. • Rosenfeld, P. , Edwards, J. E. , & Thomas, M. D. , (eds), 1993, Improving organizational surveys, SAGE Pub. • Images from MS Word Clip Art. 104

Review 105

1. The Research Process • Independent verification of results. • Designing the experiment for outcomes • Journal rankings 106

2. Literature Search • Using the web etc 107

3. Report writing, Data Collection & Presentation • • Abstract Referencing Equations Figures Conclusions and Further work Qualitative and quantitative data Plotting techniques for multi-dimensional data 108

4. Statistical Analysis and Sampling • Regression analysis • How to select a random sample. 109

5. Making a Presentation 110

6. Survey research methods • How to create and analyse a survey. 111

Why this presentation? • To develop an understanding of the scientific environment in which research is conducted. 112

Student Evaluation of Course • Survey! 113