22 tips for better data science

<div align="center" class="MsoNormal" style="background: white; line-height: 30.0pt; margin-bottom: .0001pt; margin-bottom: 0cm; mso-outline-level: 1; text-align: center; vertical-align: baseline;"><br /></div><div class="MsoNormal" style="background: white; line-height: 18.0pt; margin-bottom: .0001pt; margin-bottom: 0cm; vertical-align: baseline;"><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;">These tips are provided by </span><span style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;"><a href="http://www.datasciencecentral.com/profiles/blogs/my-data-science-journey" target="_blank"><span lang="EN-US" style="border: none windowtext 1.0pt; color: #7b539d; mso-ansi-language: EN-US; mso-border-alt: none windowtext 0cm; padding: 0cm;">Dr Granville</span></a></span><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;">, who brings 20 years of varied data-intensive experience working with successful start-ups, small companies across various industries, and eBay, Visa, Microsoft, GE and Wells Fargo.<o:p></o:p></span></div><div class="MsoNormal" style="background: white; line-height: 18.0pt; margin-bottom: 22.5pt; vertical-align: baseline;"><span style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK; mso-no-proof: yes;"><v:shapetype coordsize="21600,21600" filled="f" id="_x0000_t75" o:preferrelative="t" o:spt="75" path="m@4@5l@4@11@9@11@9@5xe" stroked="f"> <v:stroke joinstyle="miter"> <v:formulas> <v:f eqn="if lineDrawn pixelLineWidth 0"> <v:f eqn="sum @0 1 0"> <v:f eqn="sum 0 0 @1"> <v:f eqn="prod @2 1 2"> <v:f eqn="prod @3 21600 pixelWidth"> <v:f eqn="prod @3 21600 pixelHeight"> <v:f eqn="sum @0 0 1"> <v:f eqn="prod @6 1 2"> <v:f eqn="prod @7 21600 pixelWidth"> <v:f eqn="sum @8 21600 0"> <v:f eqn="prod @7 21600 pixelHeight"> <v:f eqn="sum @10 21600 0"> </v:f></v:f></v:f></v:f></v:f></v:f></v:f></v:f></v:f></v:f></v:f></v:f></v:formulas> <v:path gradientshapeok="t" o:connecttype="rect" o:extrusionok="f"> <o:lock aspectratio="t" v:ext="edit"></o:lock></v:path></v:stroke></v:shapetype><v:shape alt="https://media.licdn.com/mpr/mpr/p/5/005/099/0d4/2c29d4a.png" id="Picture_x0020_1" o:spid="_x0000_i1025" style="height: 134.4pt; mso-wrap-style: square; visibility: visible; width: 441pt;" type="#_x0000_t75"> <v:imagedata o:title="2c29d4a" src="file:///D:\Users\bismaili\AppData\Local\Temp\msohtmlclip1\01\clip_image001.jpg"></v:imagedata></v:shape></span><span style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;"><o:p></o:p></span></div><div class="MsoNormal" style="background: white; line-height: 18.0pt; margin-bottom: 3.75pt; margin-left: 30.0pt; margin-right: 0cm; margin-top: 0cm; mso-list: l0 level1 lfo1; tab-stops: list 36.0pt; text-indent: -18.0pt; vertical-align: baseline;"><!--[if !supportLists]--><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: inherit; mso-fareast-font-family: inherit; mso-fareast-language: NO-BOK;">1.<span style="font-family: 'Times New Roman'; font-size: 7pt; font-stretch: normal; line-height: normal;">     </span></span><!--[endif]--><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;">Leverage external data sources: tweets about your company or your competitors, or data from your vendors (for instance, customizable newsletter eBlast statistics available via vendor dashboards, or via submitting a ticket)<o:p></o:p></span></div><div class="MsoNormal" style="background: white; line-height: 18.0pt; margin-bottom: 3.75pt; margin-left: 30.0pt; margin-right: 0cm; margin-top: 0cm; mso-list: l0 level1 lfo1; tab-stops: list 36.0pt; text-indent: -18.0pt; vertical-align: baseline;"><!--[if !supportLists]--><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: inherit; mso-fareast-font-family: inherit; mso-fareast-language: NO-BOK;">2.<span style="font-family: 'Times New Roman'; font-size: 7pt; font-stretch: normal; line-height: normal;">     </span></span><!--[endif]--><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;">Nuclear physicists, mechanical engineers, and bioinformatics experts can make great data scientists.<o:p></o:p></span></div><div class="MsoNormal" style="background: white; line-height: 18.0pt; margin-bottom: 3.75pt; margin-left: 30.0pt; margin-right: 0cm; margin-top: 0cm; mso-list: l0 level1 lfo1; tab-stops: list 36.0pt; text-indent: -18.0pt; vertical-align: baseline;"><!--[if !supportLists]--><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: inherit; mso-fareast-font-family: inherit; mso-fareast-language: NO-BOK;">3.<span style="font-family: 'Times New Roman'; font-size: 7pt; font-stretch: normal; line-height: normal;">     </span></span><!--[endif]--><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;">State your problem correctly, and use sound metrics to measure yield provided by data science initiatives.<o:p></o:p></span></div><div class="MsoNormal" style="background: white; line-height: 18.0pt; margin-bottom: 3.75pt; margin-left: 30.0pt; margin-right: 0cm; margin-top: 0cm; mso-list: l0 level1 lfo1; tab-stops: list 36.0pt; text-indent: -18.0pt; vertical-align: baseline;"><!--[if !supportLists]--><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: inherit; mso-fareast-font-family: inherit; mso-fareast-language: NO-BOK;">4.<span style="font-family: 'Times New Roman'; font-size: 7pt; font-stretch: normal; line-height: normal;">     </span></span><!--[endif]--><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;">Use the right KPIs (key metrics) and the right data from the beginning, in any project. Changes due to bad foundations are very costly. This requires careful analysis of your data to create useful databases.<o:p></o:p></span></div><div class="MsoNormal" style="background: white; line-height: 18.0pt; margin-bottom: 3.75pt; margin-left: 30.0pt; margin-right: 0cm; margin-top: 0cm; mso-list: l0 level1 lfo1; tab-stops: list 36.0pt; text-indent: -18.0pt; vertical-align: baseline;"><!--[if !supportLists]--><span style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-bidi-font-family: inherit; mso-fareast-font-family: inherit; mso-fareast-language: NO-BOK;">5.<span style="font-family: 'Times New Roman'; font-size: 7pt; font-stretch: normal; line-height: normal;">     </span></span><!--[endif]--><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;">Fast delivery is better than extreme accuracy. All data sets are dirty anyway. </span><span style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;">Find the perfect compromise between perfection and fast return.<o:p></o:p></span></div><div class="MsoNormal" style="background: white; line-height: 18.0pt; margin-bottom: .0001pt; margin-bottom: 0cm; margin-left: 30.0pt; margin-right: 0cm; margin-top: 0cm; mso-list: l0 level1 lfo1; tab-stops: list 36.0pt; text-indent: -18.0pt; vertical-align: baseline;"><!--[if !supportLists]--><span style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-bidi-font-family: inherit; mso-fareast-font-family: inherit; mso-fareast-language: NO-BOK;">6.<span style="font-family: 'Times New Roman'; font-size: 7pt; font-stretch: normal; line-height: normal;">     </span></span><!--[endif]--><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;">With big data, strong signals (extremes) will usually be noise. </span><span style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;"><a href="http://bit.ly/Ssj8Hx" target="_blank"><span style="border: none windowtext 1.0pt; color: #7b539d; mso-border-alt: none windowtext 0cm; padding: 0cm;">Here's a solution</span></a>.<o:p></o:p></span></div><div class="MsoNormal" style="background: white; line-height: 18.0pt; margin-bottom: 3.75pt; margin-left: 30.0pt; margin-right: 0cm; margin-top: 0cm; mso-list: l0 level1 lfo1; tab-stops: list 36.0pt; text-indent: -18.0pt; vertical-align: baseline;"><!--[if !supportLists]--><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: inherit; mso-fareast-font-family: inherit; mso-fareast-language: NO-BOK;">7.<span style="font-family: 'Times New Roman'; font-size: 7pt; font-stretch: normal; line-height: normal;">     </span></span><!--[endif]--><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;">Big data has less value than useful data.<o:p></o:p></span></div><div class="MsoNormal" style="background: white; line-height: 18.0pt; margin-bottom: 3.75pt; margin-left: 30.0pt; margin-right: 0cm; margin-top: 0cm; mso-list: l0 level1 lfo1; tab-stops: list 36.0pt; text-indent: -18.0pt; vertical-align: baseline;"><!--[if !supportLists]--><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: inherit; mso-fareast-font-family: inherit; mso-fareast-language: NO-BOK;">8.<span style="font-family: 'Times New Roman'; font-size: 7pt; font-stretch: normal; line-height: normal;">     </span></span><!--[endif]--><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;">Use big data from third party vendors, for competitive intelligence.<o:p></o:p></span></div><div class="MsoNormal" style="background: white; line-height: 18.0pt; margin-bottom: .0001pt; margin-bottom: 0cm; margin-left: 30.0pt; margin-right: 0cm; margin-top: 0cm; mso-list: l0 level1 lfo1; tab-stops: list 36.0pt; text-indent: -18.0pt; vertical-align: baseline;"><!--[if !supportLists]--><span style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-bidi-font-family: inherit; mso-fareast-font-family: inherit; mso-fareast-language: NO-BOK;">9.<span style="font-family: 'Times New Roman'; font-size: 7pt; font-stretch: normal; line-height: normal;">     </span></span><!--[endif]--><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;">You can build cheap, great, scalable, robust tools pretty fast, without using </span><span style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;"><a href="http://www.datasciencecentral.com/profiles/blogs/10-types-of-regressions-which-one-to-use" target="_blank"><span lang="EN-US" style="border: none windowtext 1.0pt; color: #7b539d; mso-ansi-language: EN-US; mso-border-alt: none windowtext 0cm; padding: 0cm;">old-fashioned statistical science</span></a></span><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;">. </span><span style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;">Think about <a href="http://www.datasciencecentral.com/profiles/blogs/black-box-confidence-intervals-excel-and-perl-implementations-det" target="_blank"><span style="border: none windowtext 1.0pt; color: #7b539d; mso-border-alt: none windowtext 0cm; padding: 0cm;">model-free techniques</span></a>.<o:p></o:p></span></div><div class="MsoNormal" style="background: white; line-height: 18.0pt; margin-bottom: .0001pt; margin-bottom: 0cm; margin-left: 30.0pt; margin-right: 0cm; margin-top: 0cm; mso-list: l0 level1 lfo1; tab-stops: list 36.0pt; text-indent: -18.0pt; vertical-align: baseline;"><!--[if !supportLists]--><span style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-bidi-font-family: inherit; mso-fareast-font-family: inherit; mso-fareast-language: NO-BOK;">10.<span style="font-family: 'Times New Roman'; font-size: 7pt; font-stretch: normal; line-height: normal;">  </span></span><!--[endif]--><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;">Big data is easier and </span><span style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;"><a href="http://www.datasciencecentral.com/profiles/blogs/big-data-is-cheap-and-easy" target="_blank"><span lang="EN-US" style="border: none windowtext 1.0pt; color: #7b539d; mso-ansi-language: EN-US; mso-border-alt: none windowtext 0cm; padding: 0cm;">less costly than you think</span></a></span><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;">. </span><span style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;">Get the right tools! <a href="http://www.datasciencecentral.com/forum/topics/how-to-choose-an-analytic-tool" target="_blank"><span style="border: none windowtext 1.0pt; color: #7b539d; mso-border-alt: none windowtext 0cm; padding: 0cm;">Here's how to get started</span></a>.<o:p></o:p></span></div><div class="MsoNormal" style="background: white; line-height: 18.0pt; margin-bottom: .0001pt; margin-bottom: 0cm; margin-left: 30.0pt; margin-right: 0cm; margin-top: 0cm; mso-list: l0 level1 lfo1; tab-stops: list 36.0pt; text-indent: -18.0pt; vertical-align: baseline;"><!--[if !supportLists]--><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: inherit; mso-fareast-font-family: inherit; mso-fareast-language: NO-BOK;">11.<span style="font-family: 'Times New Roman'; font-size: 7pt; font-stretch: normal; line-height: normal;">  </span></span><!--[endif]--><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;">Correlation is not causation. </span><span style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;"><a href="http://www.datasciencecentral.com/profiles/blogs/tutorial-how-to-detect-spurious-correlations-and-how-to-find-the-" target="_blank"><span lang="EN-US" style="border: none windowtext 1.0pt; color: #7b539d; mso-ansi-language: EN-US; mso-border-alt: none windowtext 0cm; padding: 0cm;">This article</span></a></span><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;"> might help you with this issue.<o:p></o:p></span></div><div class="MsoNormal" style="background: white; line-height: 18.0pt; margin-bottom: .0001pt; margin-bottom: 0cm; margin-left: 30.0pt; margin-right: 0cm; margin-top: 0cm; mso-list: l0 level1 lfo1; tab-stops: list 36.0pt; text-indent: -18.0pt; vertical-align: baseline;"><!--[if !supportLists]--><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: inherit; mso-fareast-font-family: inherit; mso-fareast-language: NO-BOK;">12.<span style="font-family: 'Times New Roman'; font-size: 7pt; font-stretch: normal; line-height: normal;">  </span></span><!--[endif]--><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;">You don't have to store all your data permanently. Use smart compression techniques, and keep statistical summaries only, for old data. Don't forget to adjust your metrics when your data changes,</span><span style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;"><a href="http://www.freshpatents.com/-dt20110630ptan20110161492.php" target="_blank"><span lang="EN-US" style="border: none windowtext 1.0pt; color: #7b539d; mso-ansi-language: EN-US; mso-border-alt: none windowtext 0cm; padding: 0cm;">to keep consistency for trending purposes</span></a></span><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;">.<o:p></o:p></span></div><div class="MsoNormal" style="background: white; line-height: 18.0pt; margin-bottom: .0001pt; margin-bottom: 0cm; margin-left: 30.0pt; margin-right: 0cm; margin-top: 0cm; mso-list: l0 level1 lfo1; tab-stops: list 36.0pt; text-indent: -18.0pt; vertical-align: baseline;"><!--[if !supportLists]--><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: inherit; mso-fareast-font-family: inherit; mso-fareast-language: NO-BOK;">13.<span style="font-family: 'Times New Roman'; font-size: 7pt; font-stretch: normal; line-height: normal;">  </span></span><!--[endif]--><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;">A lot can be done without databases, </span><span style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;"><a href="http://www.datasciencecentral.com/group/research/forum/topics/practical-illustration-of-map-reduce-hadoop-style-on-real-data" target="_blank"><span lang="EN-US" style="border: none windowtext 1.0pt; color: #7b539d; mso-ansi-language: EN-US; mso-border-alt: none windowtext 0cm; padding: 0cm;">especially for big data</span></a></span><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;">.<o:p></o:p></span></div><div class="MsoNormal" style="background: white; line-height: 18.0pt; margin-bottom: .0001pt; margin-bottom: 0cm; margin-left: 30.0pt; margin-right: 0cm; margin-top: 0cm; mso-list: l0 level1 lfo1; tab-stops: list 36.0pt; text-indent: -18.0pt; vertical-align: baseline;"><!--[if !supportLists]--><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: inherit; mso-fareast-font-family: inherit; mso-fareast-language: NO-BOK;">14.<span style="font-family: 'Times New Roman'; font-size: 7pt; font-stretch: normal; line-height: normal;">  </span></span><!--[endif]--><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;">Always include EDA and DOE (exploratory analysis / design of experiment) early on in any data science projects. Always create a </span><span style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;"><a href="http://www.analyticbridge.com/profiles/blogs/why-and-how-you-should-build-a-data-dictionary-for-big-data-sets" target="_blank"><span lang="EN-US" style="border: none windowtext 1.0pt; color: #7b539d; mso-ansi-language: EN-US; mso-border-alt: none windowtext 0cm; padding: 0cm;">data dictionary</span></a></span><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;">. And follow the traditional </span><span style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;"><a href="http://www.datasciencecentral.com/profiles/blogs/life-cycle-of-data-science-projects" target="_blank"><span lang="EN-US" style="border: none windowtext 1.0pt; color: #7b539d; mso-ansi-language: EN-US; mso-border-alt: none windowtext 0cm; padding: 0cm;">life cycle of any data science project</span></a></span><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;">.<o:p></o:p></span></div><div class="MsoNormal" style="background: white; line-height: 18.0pt; margin-bottom: 3.75pt; margin-left: 30.0pt; margin-right: 0cm; margin-top: 0cm; mso-list: l0 level1 lfo1; tab-stops: list 36.0pt; text-indent: -18.0pt; vertical-align: baseline;"><!--[if !supportLists]--><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: inherit; mso-fareast-font-family: inherit; mso-fareast-language: NO-BOK;">15.<span style="font-family: 'Times New Roman'; font-size: 7pt; font-stretch: normal; line-height: normal;">  </span></span><!--[endif]--><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;">Data can be used for many purposes:<o:p></o:p></span></div><div class="MsoNormal" style="background: white; line-height: 18.0pt; margin-bottom: 3.75pt; margin-left: 60.0pt; margin-right: 0cm; margin-top: 0cm; mso-list: l0 level2 lfo1; tab-stops: list 72.0pt; text-indent: -18.0pt; vertical-align: baseline;"><!--[if !supportLists]--><span style="color: #4d4f51; font-family: Symbol; font-size: 10.0pt; mso-bidi-font-family: Symbol; mso-bidi-font-size: 12.0pt; mso-fareast-font-family: Symbol; mso-fareast-language: NO-BOK;">·<span style="font-family: 'Times New Roman'; font-size: 7pt; font-stretch: normal; line-height: normal;">        </span></span><!--[endif]--><span style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;">quality assurance<o:p></o:p></span></div><div class="MsoNormal" style="background: white; line-height: 18.0pt; margin-bottom: 3.75pt; margin-left: 60.0pt; margin-right: 0cm; margin-top: 0cm; mso-list: l0 level2 lfo1; tab-stops: list 72.0pt; text-indent: -18.0pt; vertical-align: baseline;"><!--[if !supportLists]--><span lang="EN-US" style="color: #4d4f51; font-family: Symbol; font-size: 10.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Symbol; mso-bidi-font-size: 12.0pt; mso-fareast-font-family: Symbol; mso-fareast-language: NO-BOK;">·<span style="font-family: 'Times New Roman'; font-size: 7pt; font-stretch: normal; line-height: normal;">        </span></span><!--[endif]--><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;">to find actionable patterns (stock trading, fraud detection)<o:p></o:p></span></div><div class="MsoNormal" style="background: white; line-height: 18.0pt; margin-bottom: 3.75pt; margin-left: 60.0pt; margin-right: 0cm; margin-top: 0cm; mso-list: l0 level2 lfo1; tab-stops: list 72.0pt; text-indent: -18.0pt; vertical-align: baseline;"><!--[if !supportLists]--><span lang="EN-US" style="color: #4d4f51; font-family: Symbol; font-size: 10.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Symbol; mso-bidi-font-size: 12.0pt; mso-fareast-font-family: Symbol; mso-fareast-language: NO-BOK;">·<span style="font-family: 'Times New Roman'; font-size: 7pt; font-stretch: normal; line-height: normal;">        </span></span><!--[endif]--><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;">for resale to your business clients<o:p></o:p></span></div><div class="MsoNormal" style="background: white; line-height: 18.0pt; margin-bottom: 3.75pt; margin-left: 60.0pt; margin-right: 0cm; margin-top: 0cm; mso-list: l0 level2 lfo1; tab-stops: list 72.0pt; text-indent: -18.0pt; vertical-align: baseline;"><!--[if !supportLists]--><span lang="EN-US" style="color: #4d4f51; font-family: Symbol; font-size: 10.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Symbol; mso-bidi-font-size: 12.0pt; mso-fareast-font-family: Symbol; mso-fareast-language: NO-BOK;">·<span style="font-family: 'Times New Roman'; font-size: 7pt; font-stretch: normal; line-height: normal;">        </span></span><!--[endif]--><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;">to optimize decisions and processes (operations research)<o:p></o:p></span></div><div class="MsoNormal" style="background: white; line-height: 18.0pt; margin-bottom: 3.75pt; margin-left: 60.0pt; margin-right: 0cm; margin-top: 0cm; mso-list: l0 level2 lfo1; tab-stops: list 72.0pt; text-indent: -18.0pt; vertical-align: baseline;"><!--[if !supportLists]--><span lang="EN-US" style="color: #4d4f51; font-family: Symbol; font-size: 10.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Symbol; mso-bidi-font-size: 12.0pt; mso-fareast-font-family: Symbol; mso-fareast-language: NO-BOK;">·<span style="font-family: 'Times New Roman'; font-size: 7pt; font-stretch: normal; line-height: normal;">        </span></span><!--[endif]--><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;">for investigation and discovery (IRS, litigation, fraud detection, root cause analysis)<o:p></o:p></span></div><div class="MsoNormal" style="background: white; line-height: 18.0pt; margin-bottom: 3.75pt; margin-left: 60.0pt; margin-right: 0cm; margin-top: 0cm; mso-list: l0 level2 lfo1; tab-stops: list 72.0pt; text-indent: -18.0pt; vertical-align: baseline;"><!--[if !supportLists]--><span lang="EN-US" style="color: #4d4f51; font-family: Symbol; font-size: 10.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Symbol; mso-bidi-font-size: 12.0pt; mso-fareast-font-family: Symbol; mso-fareast-language: NO-BOK;">·<span style="font-family: 'Times New Roman'; font-size: 7pt; font-stretch: normal; line-height: normal;">        </span></span><!--[endif]--><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;">machine-to-machine communication (automated bidding systems, automated driving)<o:p></o:p></span></div><div class="MsoNormal" style="background: white; line-height: 18.0pt; margin-bottom: 3.75pt; margin-left: 60.0pt; margin-right: 0cm; margin-top: 0cm; mso-list: l0 level2 lfo1; tab-stops: list 72.0pt; text-indent: -18.0pt; vertical-align: baseline;"><!--[if !supportLists]--><span lang="EN-US" style="color: #4d4f51; font-family: Symbol; font-size: 10.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Symbol; mso-bidi-font-size: 12.0pt; mso-fareast-font-family: Symbol; mso-fareast-language: NO-BOK;">·<span style="font-family: 'Times New Roman'; font-size: 7pt; font-stretch: normal; line-height: normal;">        </span></span><!--[endif]--><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;">predictions (sales forecasts, growth and financial predictions, weather)<o:p></o:p></span></div><div class="MsoNormal" style="background: white; line-height: 18.0pt; margin-bottom: .0001pt; margin-bottom: 0cm; margin-left: 30.0pt; margin-right: 0cm; margin-top: 0cm; mso-list: l0 level1 lfo1; tab-stops: list 36.0pt; text-indent: -18.0pt; vertical-align: baseline;"><!--[if !supportLists]--><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: inherit; mso-fareast-font-family: inherit; mso-fareast-language: NO-BOK;">16.<span style="font-family: 'Times New Roman'; font-size: 7pt; font-stretch: normal; line-height: normal;">  </span></span><!--[endif]--><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;">Don't dump Excel. </span><span style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;"><a href="http://www.datasciencecentral.com/profiles/blogs/sample-data-science-project-optimizing-all-business-levers-simult" target="_blank"><span lang="EN-US" style="border: none windowtext 1.0pt; color: #7b539d; mso-ansi-language: EN-US; mso-border-alt: none windowtext 0cm; padding: 0cm;">Embrace light analytics</span></a></span><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;">.<o:p></o:p></span></div><div class="MsoNormal" style="background: white; line-height: 18.0pt; margin-bottom: 3.75pt; margin-left: 30.0pt; margin-right: 0cm; margin-top: 0cm; mso-list: l0 level1 lfo1; tab-stops: list 36.0pt; text-indent: -18.0pt; vertical-align: baseline;"><!--[if !supportLists]--><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: inherit; mso-fareast-font-family: inherit; mso-fareast-language: NO-BOK;">17.<span style="font-family: 'Times New Roman'; font-size: 7pt; font-stretch: normal; line-height: normal;">  </span></span><!--[endif]--><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;">Data + models + gut feelings + intuition is the perfect mix. Don't remove any of these ingredients in your decision process.<o:p></o:p></span></div><div class="MsoNormal" style="background: white; line-height: 18.0pt; margin-bottom: .0001pt; margin-bottom: 0cm; margin-left: 30.0pt; margin-right: 0cm; margin-top: 0cm; mso-list: l0 level1 lfo1; tab-stops: list 36.0pt; text-indent: -18.0pt; vertical-align: baseline;"><!--[if !supportLists]--><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: inherit; mso-fareast-font-family: inherit; mso-fareast-language: NO-BOK;">18.<span style="font-family: 'Times New Roman'; font-size: 7pt; font-stretch: normal; line-height: normal;">  </span></span><!--[endif]--><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;">Leverage the power of compound metrics: KPIs derived from database fields, that have a far better</span><a href="http://www.datasciencecentral.com/profiles/blogs/feature-selection-based-on-predictive-power" target="_blank"><span lang="EN-US" style="border: none windowtext 1.0pt; color: #7b539d; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Helvetica; mso-border-alt: none windowtext 0cm; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK; padding: 0cm;">predictive power</span></a><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;"> than the original database metrics. For instance your database might include a single keyword field but does not discriminate between user query and search category (sometimes because data comes from various sources and is blended together). Detect the issue, and create a new metric called keyword type - or data source. Another example is </span><a href="http://www.datasciencecentral.com/group/research/forum/topics/internet-topology-mapping" target="_blank"><span lang="EN-US" style="border: none windowtext 1.0pt; color: #7b539d; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Helvetica; mso-border-alt: none windowtext 0cm; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK; padding: 0cm;">IP address category</span></a><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;">, a fundamental metric that should be created and added to all digital analytics projects.<o:p></o:p></span></div><div class="MsoNormal" style="background: white; line-height: 18.0pt; margin-bottom: .0001pt; margin-bottom: 0cm; margin-left: 30.0pt; margin-right: 0cm; margin-top: 0cm; mso-list: l0 level1 lfo1; tab-stops: list 36.0pt; text-indent: -18.0pt; vertical-align: baseline;"><!--[if !supportLists]--><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: inherit; mso-fareast-font-family: inherit; mso-fareast-language: NO-BOK;">19.<span style="font-family: 'Times New Roman'; font-size: 7pt; font-stretch: normal; line-height: normal;">  </span></span><!--[endif]--><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;">When do you need true real time processing? When fraud detection is critical, or when processing sensitive transactional data (credit card fraud detection, 911 calls). Other than that, <i><span style="border: none windowtext 1.0pt; mso-border-alt: none windowtext 0cm; padding: 0cm;">delayed analytics</span></i>(with a latency of a few seconds to 24 hours) is good enough.<o:p></o:p></span></div><div class="MsoNormal" style="background: white; line-height: 18.0pt; margin-bottom: .0001pt; margin-bottom: 0cm; margin-left: 30.0pt; margin-right: 0cm; margin-top: 0cm; mso-list: l0 level1 lfo1; tab-stops: list 36.0pt; text-indent: -18.0pt; vertical-align: baseline;"><!--[if !supportLists]--><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: inherit; mso-fareast-font-family: inherit; mso-fareast-language: NO-BOK;">20.<span style="font-family: 'Times New Roman'; font-size: 7pt; font-stretch: normal; line-height: normal;">  </span></span><!--[endif]--><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;">Make sure your sensitive data is well protected. Make sure your algorithms can not be tampered by criminal hackers or business hackers (spying on your business and stealing everything they can, legally or illegally, and jeopardizing your algorithms - which translates in severe revenue loss). An example of business hacking can be found in section 3 i</span><a href="http://www.datasciencecentral.com/profiles/blogs/could-fake-reviews-kill-amazon" target="_blank"><span lang="EN-US" style="border: none windowtext 1.0pt; color: #7b539d; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Helvetica; mso-border-alt: none windowtext 0cm; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK; padding: 0cm;">n this article</span></a><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;">.<o:p></o:p></span></div><div class="MsoNormal" style="background: white; line-height: 18.0pt; margin-bottom: .0001pt; margin-bottom: 0cm; margin-left: 30.0pt; margin-right: 0cm; margin-top: 0cm; mso-list: l0 level1 lfo1; tab-stops: list 36.0pt; text-indent: -18.0pt; vertical-align: baseline;"><!--[if !supportLists]--><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: inherit; mso-fareast-font-family: inherit; mso-fareast-language: NO-BOK;">21.<span style="font-family: 'Times New Roman'; font-size: 7pt; font-stretch: normal; line-height: normal;">  </span></span><!--[endif]--><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;">Blend multiple models together to detect many types of patterns. Average these models. </span><a href="http://www.datasciencecentral.com/profiles/blogs/hidden-decision-trees-revisited" target="_blank"><span lang="EN-US" style="border: none windowtext 1.0pt; color: #7b539d; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Helvetica; mso-border-alt: none windowtext 0cm; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK; padding: 0cm;">Here's a simple example of model blending</span></a><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;">.<o:p></o:p></span></div><div class="MsoNormal" style="background: white; line-height: 18.0pt; margin-bottom: .0001pt; margin-bottom: 0cm; margin-left: 30.0pt; margin-right: 0cm; margin-top: 0cm; mso-list: l0 level1 lfo1; tab-stops: list 36.0pt; text-indent: -18.0pt; vertical-align: baseline;"><!--[if !supportLists]--><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: inherit; mso-fareast-font-family: inherit; mso-fareast-language: NO-BOK;">22.<span style="font-family: 'Times New Roman'; font-size: 7pt; font-stretch: normal; line-height: normal;">  </span></span><!--[endif]--><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;">Ask </span><a href="http://www.datasciencecentral.com/forum/topics/how-to-choose-an-analytic-tool" target="_blank"><span lang="EN-US" style="border: none windowtext 1.0pt; color: #7b539d; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Helvetica; mso-border-alt: none windowtext 0cm; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK; padding: 0cm;">the right questions</span></a><span lang="EN-US" style="color: #4d4f51; font-family: "inherit","serif"; font-size: 12.0pt; mso-ansi-language: EN-US; mso-bidi-font-family: Helvetica; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: NO-BOK;"> before purchasing software.<o:p></o:p></span></div><br /><div class="MsoNormal"><br /></div>

22 tips for better data science

These tips are provided by  Dr Granville , who brings 20 years of varied data-intensive experience working with successful start-ups, small ...

Read more »

Top 10 Big Data Technologies Of Present Times

<table border="0" cellpadding="0" cellspacing="0" style="background-color: white; color: #808285; font-family: Arial, Helvetica, sans-serif; font-size: 12px; width: 100%px;"><tbody><tr><td class="td_nofeed" style="border-bottom-color: rgb(143, 0, 0); border-bottom-style: solid; border-bottom-width: 2px; height: 30px;"><span class="header" style="color: #8f0000; font-family: Georgia; font-size: 21px; font-weight: bold;"> </span> </td></tr><tr><td><table border="0" cellpadding="0" cellspacing="0" style="width: 100%px;"><tbody><tr><td> </td></tr><tr><td><table border="0" cellpadding="0" cellspacing="0" style="width: 100%px;"><tbody><tr><td valign="top" width="95%"><span style="font-family: verdana; font-size: x-small;"><b>Over last few years Big Data technologies are getting due attention and there are several trends and innovations in this space in recent times.</b> </span></td><td valign="top" width="2%"> </td><td valign="top" width="23%"></td></tr><tr><td class="font_sml" colspan="3" style="color: #808285; font-family: arial; font-size: 10px;"><div id="aj" style="cursor: pointer;"></div><span class="text2"></span></td></tr></tbody></table></td></tr><tr><td><table><tbody><tr><td valign="top"><strong><span style="color: black; font-family: verdana; font-size: x-small;">Wednesday, October 22, 2014</span></strong>:  <span style="color: black; font-family: verdana; font-size: x-small;">Big Data is a concept which is quite broad and comprises several trends and technology developments. Over last few years Big Data technologies are getting due attention and there are several trends and innovations in this space in recent times. Here we'll discuss top ten emerging Big Data technologies.<br /></span></td><td valign="top"></td></tr></tbody></table><br /><div align="center"></div><span style="color: black; font-family: verdana; font-size: x-small;"><b>1. Column-oriented databases: </b><br /><br />Traditional databases are excellent in online transaction processing but when it comes to query performance while data volumes grow, these databases fall short on performance. The new column-oriented databases store data and focuses on columns and not rows. It allows huge data compression and faster query times.<br /><br /><b>2. Streaming Big Data analytics: </b><br /><br />There are several projects in this section including Storm, Spark, Data Torrent, Spring XD and SQL Stream. Apache Storm is an open source distributed real-time computation system which simplifies streams of data and real-time processing. Spark is a data processing platform which is compatible with Hadoop. DataTorrent is a real-time streaming platform which enables businesses to perform data processing. Spring XD supports streams for event driven data while SQLStream provides a distributed stream processing platform for streaming analytics, visualization and continuous integration of machine data.<br /><br /><b>3. Schema-less databases, or NoSQL databases: </b><br /><br />This database category includes key-value stores and document stores. This database focuses on storage and retrieval of large volumes of unstructured, semi-structured or even structured data.<br /><br /><b>4. SQL-in-Hadoop: </b><br /><br />This technology includes Apache Hive, Shark, Apache Drill, Presto and Phoenix among many others. It helps in making queries and it also manages large datasets in distributed storage. Shark is a data warehouse system which supports Hive's query language. Apache Drill is an Apache incubation project and it's designed for scalability. It's backed by MapR. Presto is an open source distributed SQL query engine and Phoenix is an open source SQL query engine for Apache Hbase.<br /><br /><b>5. MapReduce: </b><br /><br />It's a programming paradigm which allows massive job execution scalability against thousands of servers or clusters of servers. Its two tasks are Map task and Reduce task. It converts any input dataset into different set of value pairs while reducing set of tuples.<br /><br /><b>6. Hadoop: </b><br /><br />Hadoop is an open source platform for handling Big Data which can work with multiple data sources. It has other applications too and it's largely used for changing data like location-based data from weather or traffic sensors, web-based or social media data or machine-to-machine transactional data.<br /><br /><b>7. PIG: </b><br /><br />PIG brings the Hadoop project close to developers and business users and it's used by Perl like language allowing query execution over data stored on a Hadoop cluster. PIG was a project by Yahoo! But now it's completely open source.<br /><br /><b>8. Big Data Lambda Architecture: </b><br /><br />Lambda Architecture is a hybrid platform which combines real-time data and pre-computed data to provide a near-real time view of the data at all times. Its frameworks include Summingbird by Twitter and Lambdoop.<br /><br /><b>9. PLATFORA: </b><br /><br />It almost copies Hadoop and it requires developer knowledge to operate. It's a platform which turns queries into Hadoop jobs with immediate effect and creates an abstraction layer to simplify the datasets in Hadoop.<br /><br /><b>10. SkyTree: </b><br /><br />It's a high performance machine learning data analytics platform which handles Big Data. It's an essential part of Big Data.<br /><br /><b>Courtesy: TechRepublic and InfoQ </b><br /><br /><span style="color: black;"><b><i>Sanchari Banerjee, EFYTIMES News Network</i></b></span> </span></td></tr></tbody></table></td></tr></tbody></table>

Top 10 Big Data Technologies Of Present Times

      Over last few years Big Data technologies are getting due attention and there are several trends and innovations in this space in rece...

Read more »