Open Source Intelligence Tools And Resources Handbook 2020


Download Open Source Intelligence Tools And Resources Handbook 2020


Preview text

OPEN SOURCE INTELLIGENCE TOOLS AND RESOURCES HANDBOOK
2020

OPEN SOURCE INTELLIGENCE TOOLS AND RESOURCES HANDBOOK
2020
Aleksandra Bielska
Noa Rebecca Kurz, Yves Baumgartner, Vytenis Benetis
2

Foreword
I am delighted to share with you the 2020 edition of the OSINT Tools and Resources Handbook. Once again, the Handbook has been revised and updated to reflect the evolution of this discipline, and the many strategic, operational and technical challenges OSINT practitioners have to grapple with.
Given the speed of change on the web, some might question the wisdom of pulling together such a resource. What’s wrong with the Top 10 tools, or the Top 100? There are only so many resources one can bookmark after all. Such arguments are not without merit. My fear, however, is that they are also shortsighted. I offer four reasons why.
To begin, a shortlist betrays the widening spectrum of OSINT practice. Whereas OSINT was once the preserve of analysts working in national security, it now embraces a growing class of professionals in fields as diverse as journalism, cybersecurity, investment research, crisis management and human rights. A limited toolkit can never satisfy all of these constituencies.
Second, a good OSINT practitioner is someone who is comfortable working with different tools, sources and collection strategies. The temptation toward narrow specialisation in OSINT is one that has to be resisted. Why? Because no research task is ever as tidy as the customer’s requirements are likely to suggest.
Third, is the inevitable realisation that good tool awareness is equivalent to good source awareness. Indeed, the right tool can determine whether you harvest the right information. It follows that the more tools you have in your portfolio, the more flexible your OSINT capabilities are likely to be.
Finally, the process of compiling this handbook is an intelligence exercise in its own right, alerting us to where we are at as a community and the challenges we are likely to face going forward – not least of which are disinformation, the fracturing of the internet, the proliferation of niche social media platforms, and the urgent need for better tools to monitor and analyse the content they provide.
Our hope is that this Handbook helps you plug any gaps in your collection efforts and alerts you to the many tools and resources you can consult for better intelligence, including on the discipline of OSINT itself. Of course, the Handbook makes no claim to being complete. Something new will doubtless pop-up tomorrow, and I suspect there are hundreds of tools we haven’t even come across at all. With your help, we can address any omissions.
In any event, the usual caveats apply. While every effort has been made to test and validate these resources, we caution that today’s wonder tool may be tomorrow’s security liability. Thus, whenever testing a new script, application or browser extension, please do so in a secure environment to minimise any risk to your assets or operations.
3

I would like to thank my colleagues at i-intelligence for their efforts in compiling this Handbook. Much of the credit belongs to Aleksandra Bielska who, as our Head of Training, evaluates hundreds of tools every year to determine which ones make it into our courses. I am also indebted to Yves Baumgartner and Vytenis Benetis for routinely updating us on new solutions. Thanks too are also due to Noa Kurz who worked tirelessly over the summer to help us get this document ready. I close by acknowledging the contributions of the many students, practitioners and Twitter followers we’ve interacted with over the past two years. Your generosity, insights, suggestions and support are much appreciated. Chris Pallaris Director, i-intelligence GmbH
4

Table of Contents
Foreword............................................................................................................................. 3 Search Tools ..................................................................................................................... 17
General Search .......................................................................................................................... 17 Meta Search ............................................................................................................................... 18 Visual Search and Clustering Search Engines ......................................................................... 20 Decentralised Search Engines.................................................................................................. 21 Charitable Search Engines........................................................................................................ 22 Code Search .............................................................................................................................. 23 File and FTP Search................................................................................................................... 25 Open Database, Directories and Services Search .................................................................. 26 Similar Sites Search ................................................................................................................... 27 Children-Friendly Search Engines............................................................................................ 28 National Search Engine Directories......................................................................................... 29 Custom Search........................................................................................................................... 30 Specialty Search Engines .......................................................................................................... 31 Search Tools............................................................................................................................... 32 Social Media ..................................................................................................................... 34 Major Social Networks .............................................................................................................. 34 Social Media Search and Monitoring ...................................................................................... 37 Working with Hashtags ............................................................................................................. 39 Finding Influencers and Other Social Media Users ................................................................ 41 Social Media Management and Content Discovery............................................................... 42 Social Media Analytics .............................................................................................................. 43 Social Media Tools: Twitter ...................................................................................................... 44 Social Media Tools: Facebook ................................................................................................. 49 Social Media Tools: Instagram ................................................................................................. 51
5

Social Media Tools: Pinterest ................................................................................................... 55 Social Media Tools: Reddit ....................................................................................................... 56 Social Media Tools: Voat .......................................................................................................... 59 Social Media Tools: VKontakte................................................................................................. 60 Social Media Tools: LinkedIn.................................................................................................... 61 Social Media Tools: Github ...................................................................................................... 63 Social Media Tools: TikTok ....................................................................................................... 65 Messaging and Chat Tools ....................................................................................................... 66 Social Media Tools: Snapchat .................................................................................................. 68 Social Media Tools: WhatsApp ................................................................................................ 69 Social Media Tools: Skype ........................................................................................................ 70 Social Media Tools: Kik ............................................................................................................. 71 Social Media Tools: Telegram .................................................................................................. 72 Social Media Tools: IRC ............................................................................................................ 74 Dating Apps and Sites .............................................................................................................. 75 Fitness Apps and Communities ............................................................................................... 76 Working with Blogs .......................................................................................................... 77 Blog Search ................................................................................................................................ 77 Working with Forums ....................................................................................................... 78 Forum, Newsgroup and Discussion Board Search Tools....................................................... 78 Working with Chan Boards.............................................................................................. 79 Chan Search Tools..................................................................................................................... 79 Working with Q&A Sites .................................................................................................. 80 Q&A Sites................................................................................................................................... 80 People Investigations....................................................................................................... 81 People Search Tools ................................................................................................................. 81 Name Research.......................................................................................................................... 84 Public Records ........................................................................................................................... 85 Document Checking ................................................................................................................. 87 Ancestry Research ..................................................................................................................... 88 Missing People .......................................................................................................................... 89 Address and Contact Information Search ............................................................................... 91 Contact Detail Recovery Apps ................................................................................................. 93
6

Investigating Phone Numbers.................................................................................................. 95 Cell Tower Search...................................................................................................................... 99 E-mail Search ........................................................................................................................... 100 Validating E-mails .................................................................................................................... 102 Analysing E-mails..................................................................................................................... 104 Analysing E-mail Headers ....................................................................................................... 105 Matching E-mails to Social Media Profiles ............................................................................ 106 IP Grabbers and E-mail Trackers ............................................................................................ 107 Username Check ..................................................................................................................... 108 CV and Resume Search ........................................................................................................... 110 Expert Search........................................................................................................................... 111 Business Research .......................................................................................................... 113 Company Search Tools ........................................................................................................... 113 Business Registers Lists and Directories................................................................................ 117 Job Search Resources ............................................................................................................. 118 Company Reviews ................................................................................................................... 120 Salary Search............................................................................................................................ 121 Patent Research ....................................................................................................................... 123 Trademark Search.................................................................................................................... 125 Tenders..................................................................................................................................... 127 Charity, NGO, and Nonprofit Research................................................................................. 129 Researching Leaks ................................................................................................................... 131 Real Estate Research ...................................................................................................... 132 Real Estate Research ............................................................................................................... 132 Online Marketplaces ...................................................................................................... 135 Classifieds and Online Marketplaces..................................................................................... 135 Cryptocurrency and Banking Research ........................................................................ 138 Cryptocurrency Research ........................................................................................................ 138 Banking and Credit Card Information ................................................................................... 141 Currency Conversion ............................................................................................................... 143 Legal Research ............................................................................................................... 144 Legal Research Tools and Resources .................................................................................... 144
7

Researching Countries ................................................................................................... 147 Country Profiles ....................................................................................................................... 147
Researching Terrorism, Crime, and Cybersecurity ...................................................... 148 Researching Terrorism ............................................................................................................ 148 Researching Crime .................................................................................................................. 151 Researching Drug Trade ......................................................................................................... 153 Researching Money Laundering and Corruption ................................................................. 154 Researching Human Trafficking ............................................................................................. 155 Requests for User Information: Information for Law Enforcement...................................... 158 Researching Cybersecurity ..................................................................................................... 159 Military Research ..................................................................................................................... 161 Crisis Alerts and Travel Advisory ............................................................................................ 163
Researching Stolen Items .............................................................................................. 164 Researching Stolen Items ....................................................................................................... 164
Transportation ................................................................................................................ 166 Airplanes, Airports and Flight Trackers ................................................................................. 166 Drones ...................................................................................................................................... 169 Cars ........................................................................................................................................... 171 Railways .................................................................................................................................... 173 Public Transport....................................................................................................................... 176 Ships ......................................................................................................................................... 178 Container and Cargo Tracking ............................................................................................... 181
Web Intelligence ............................................................................................................ 182 Researching Domains ............................................................................................................. 182 IP / IPv6 ..................................................................................................................................... 187 SEO Tools ................................................................................................................................ 191 Analysing Unique IDs .............................................................................................................. 193 Subdomain Enumeration ........................................................................................................ 194 Identifying Backlinks................................................................................................................ 195 Site Mapping Tools ................................................................................................................. 196 Identifying Publish Date.......................................................................................................... 197 Extracting Files ........................................................................................................................ 198 Analysing URLs ........................................................................................................................ 199
8

Identifying Technologies Behind Websites .......................................................................... 200 Finding SSL Certificates .......................................................................................................... 201 Checking Website Status........................................................................................................ 202 HTTPS Status ........................................................................................................................... 203 Researching Cyber Threats..................................................................................................... 204 IoT Search Engines.................................................................................................................. 208 Exploit Search Engines ........................................................................................................... 209 General Webint Tools and Toolkits ....................................................................................... 210 Multipurpose OSINT Frameworks ......................................................................................... 212 Web History and Site Capture ............................................................................................... 214 Wifi Search ............................................................................................................................... 217 Dark Web ........................................................................................................................ 218 Researching the Dark Web ..................................................................................................... 218 Alternatives to Tor ................................................................................................................... 220 Event Search ................................................................................................................... 221 Event Search ............................................................................................................................ 221 Working with Images ..................................................................................................... 222 Image Search and Reverse Image Search ............................................................................. 222 Searching within Screenshots................................................................................................. 225 Facial Recognition ................................................................................................................... 226 Logo Search and Identification Tools .................................................................................... 227 Investigating Objects and Symbols ....................................................................................... 228 Image Analysis ......................................................................................................................... 229 Stock Images............................................................................................................................ 232 Image and Photo Editing........................................................................................................ 234 Creating World Clouds ........................................................................................................... 237 OCR and Image Translation Tools ......................................................................................... 238 Other Image Processing Tools ............................................................................................... 239 Working with Videos ...................................................................................................... 240 Video Search and Reverse Video Search............................................................................... 240 Live Streaming and Webcams................................................................................................ 242 YouTube Tools......................................................................................................................... 245 General Video Downloaders and Archivers .......................................................................... 247
9

Downloading Videos from YouTube ..................................................................................... 249 Downloading Videos from Twitter ......................................................................................... 250 Downloading Videos from Facebook .................................................................................... 251 Downloading Videos from Instagram .................................................................................... 252 Downloading Videos from TikTok.......................................................................................... 253 Downloading Videos from Pinterest ...................................................................................... 254 Downloading Videos from Reddit.......................................................................................... 255 Video Players ........................................................................................................................... 256 Video Editors and Converters ................................................................................................ 257 Working with Video Metadata ............................................................................................... 259 Working with GIFs and Memes ..................................................................................... 260 Finding GIFs and Memes........................................................................................................ 260 Working with Audio ....................................................................................................... 261 Working with Radio ................................................................................................................. 261 Podcasts ................................................................................................................................... 263 Speech to Text Tools .............................................................................................................. 265 Text to Speech Tools .............................................................................................................. 266 Audio Editors ........................................................................................................................... 267 Working with Audio Metadata ............................................................................................... 268 3D Models and Designs ................................................................................................ 269 3D Models and Designs.......................................................................................................... 269 Working with Documents .............................................................................................. 270 Searching Web-based Documents ........................................................................................ 270 Searching Personal Document Collections ........................................................................... 271 Searching for Slides and Presentations ................................................................................. 272 Searching Pastebins ................................................................................................................ 273 Document and Reference Management ............................................................................... 274 PDF Management ................................................................................................................... 275 Working with Document Metadata ....................................................................................... 277 Academic Resources and Grey Literature.................................................................... 278 Academic and Grey Literature Research Tools..................................................................... 278 Medical Search Engines.......................................................................................................... 284
10

Preparing to load PDF file. please wait...

0 of 0
100%
Open Source Intelligence Tools And Resources Handbook 2020