A word about Open Source Intelligence.

[Star Wars]

Open source intelligence or called OSINT is any data and information acquired from public sources. Ever since the advent of social media, the ability for private individuals and organisations to acquire data on personal have become all the more easier. We have seen repeatedly PA officers posting all their history on LinkedIn and not to mention PA officers along with SSG men bragging about themselves on FB. I am enquiring whether we can scour the internet start our own analysis of PA officers, men and materials and create a database. Also warn our own members to be careful when posting images of our men in Uniform on this forum and any other forums.

 

[Shajida Khan]

Open source intelligence or called OSINT is any data and information acquired from public sources. Ever since the advent of social media, the ability for private individuals and organisations to acquire data on personal have become all the more easier. We have seen repeatedly PA officers posting all their history on LinkedIn and not to mention PA officers along with SSG men bragging about themselves on FB. I am enquiring whether we can scour the internet start our own analysis of PA officers, men and materials and create a database. Also warn our own members to be careful when posting images of our men in Uniform on this forum and any other forums.

You can and its quite cheap. I ordered few satellite pictures of Doklam for pretty cheap (by the standard of information contained in them). It was pretty nice. I could see what kind of deployments and structures Indian army and Chinese army build over 2016-18. It was fun.

You can also ask companies who usually crawl internet to do a crawl of facebook/twitter for you. Then you can create an index on it using Elastic Search and search for the keywords you want. You can create a time based snapshots of web pages. If you put this on a AWS account and are actually ready to pay for it, its still pretty cheap (100-200 dollars max per month). Or you can go for GCP or Azure.

If you want you can even create a bunch of crawlers yourself. Its not hard at all. The basic code for that exists in many GitHub projects. You can run it yourself with a cluster hosted on AWS/GCP/Azure.

Because social media based marketing insights and analytics went mainstream 5-years, scrapping social media is now a solved problem.

 

[Valkyrja]

I am enquiring whether we can scour the internet start our own analysis of PA officers, men and materials and create a database.

Of course you can. OSINT is a great way to build a picture of enemy capabilities. Just be weary of any of your own conclusions since it's equally easy for a foreign power to leak info to sway your conclusion. Here's a great list of some of the most common sites OSINT is gathered from by professional OSINT companies like Bellingcat.

Lol at "TweetBeaver".



If you want a blueprint of OSINT done right, take a look at HISutton, a premier sub-surface OSINT specialist. Here's a great article on the dangers of OSINT and their use against a Swedish submarine base:

H I Sutton - Covert Shores





Based on HISutton's article and the dangers it posed, the Swedes actually made changes to their naval bases, how their viewed and how they operate. A rare win for positive OSINT mitigation:

H I Sutton - Covert Shores

Timeline

• 2009: Public Webcam set up in Karlskrona showing harbor and approaches
• 23rd July 2017: Swedish blogger and author Lars Wilderang writes a blog post highlighting that one of the public webcamsin the Karlskrona provides a view of naval activities. Wilderang's blog covers finance, economics, environment and politics, and is one of Sweden's most read blogs.
• 24th July 2017: local Swedish newspaper Blekinge Läns Tidning (BLT) runs the story, reporting that the Swedish Navy has asked the local government to turn off or move the camera but to no avail.
• 26th July 2017: Russian state media Sputnik News runs the story ('Twice Shy' Swedish Navy Base Vexed by Archipelago Webcam Revealing Too Much)
• 5th August 2017: Covert Shores Article on use of Public Webcams to monitor submarine activity published, citing the BLT article and using the Karlskrona webcam as an example. This is one of the first (if not THE first) web articles discussing the ways in which public webcams can be used to gather operational data on submarine capabilities, and demonstrates the relevance of these cameras to OSINT
• 17th August 2017, 6am local time: Blekinge Läns Tidning publishes a follow-up article article citing H I Sutton / Covert Shores
• 17th August 2017, morning: Other Swedish media pick up on the story, including Radio Sweden.
• 17th August 2017, morning: The local government holds a meeting with the Navy to discuss the placement of the webcams. Among those attending is Swedish Defense expert Johan Wiktorin. The local government’s view on the matter is reversed and it is agreed to move the camera so that it no longer overlooks the most sensitive areas of the naval base
• 17th August 2017, midday: The camera is moved
• 17th August 2017, afternoon: Blekinge Läns Tidning reports that the camera was moved.

His articles provide some of the best insights into otherwise clandestine programs and projects, such as this article on the Russian spycraft Yantar:

H I Sutton - Covert Shores



If you're interested in OSINT, he provides a great blueprint throughout his blog and contributions to Janes on how to do it.