The Terrifying Technology Inside Drone Cameras

This episode is brought to you by the brilliant We've all seen the movie clichés of spy satellites, these all-seeing black eyes floating in space that can apparently find and track a vehicle by its license plate anywhere on Earth in a matter of seconds . However, this mystique is easily surpassed by the reality of an orbiting satellite; imagine looking at an object the size of a small car from a distance of up to 2,000 kilometers or approximately 1,200 miles through a telescope as it moves at seven kilometers and half or about 4.7 miles per second at At these relative speeds, a near-Earth satellite would easily sweep over a car-sized object in a millisecond.

Surveillance in low Earth orbit is for the most part limited by the short period of time that the system's optics are exposed to the target, as well as the limits of In the chosen orbit, the alternative to this approach is to use a network of beacons. geostationary surveillance satellites or slow-orbiting geosynchronous satellites that allow much greater orientation persistence, although this comes at the cost of significantly reduced image resolution, as these systems must now orbit between twenty thousand kilometers to 36 000 kilometers or approximately twelve thousand to twenty-two thousand miles from the surface, this tends to limit its use to more general levels.

Optical surveillance and radiation detection are also relatively expensive to operate and complex to establish, as multiple satellites are needed for global surveillance. Coverage with program costs easily approaching $1 trillion and taking more than a decade to get fully operational long before satellite reconnaissance and surveillance aircraft were the tool of choice for collecting aerial imagery and, as Satellite systems grew in sophistication, slowly evolving into a faster response tactical complement. to satellites, however similar to low-Earth orbit satellites, high-resolution imaging came at the cost of shorter optical exposure times on the target and the inherent risk to aircrew that required missions to were very specific and purposeful in nature in the 1990s, the emergence of unmanned aerial vehicles would dramatically change the nature of aerial surveillance.

UAVs could now loiter over large areas of land for several days, and within a decade, a revolution in aerial imagery would combine with these incredible UAV loitering times to create a capability that would even replace hyperbole. On film, the ability to observe every event within an entire city in real time, UAVs operate in the world of tactical intelligence surveillance and reconnaissance or ISR, generally providing immediate support for military operations, often within ever-evolving mission objectives. Traditionally, airborne ISR imaging systems were designed around one of two objectives, either looking at a larger area without the ability to provide detailed resolution of a particular object or providing a high-resolution view of a specific target with a greatly diminished ability to see the broader context well into the 1990s, traditional wet film was still a large part of ISR missions, both the SR-71 reconnaissance aircraft and the U2, employed a system called a bar camera optics that rotated film through a roller cage in a back-and-forth motion to capture wide panoramic photographs using a 5-inch or 12.7-centimeter roll of film. wide and nearly 3.2 kilometers or 2 miles long, the system would capture one frame every 6.8 seconds with a limit of about 16,000 frame grabbers per roll.

Each frame would cover a land area of ​​approximately 112 square nautical miles in a Horizons Horizon panoramic sweep. The size of each frame of the film allowed details to be resolved down to 15 centimeters or about 6 inches from 24 kilometers or about 15 miles high. Narrower field reconnaissance systems that could fly closer and capture higher resolution images of targets were deployed as reconnaissance pods for existing aircraft. The ASD2, for example, was a film-based reconnaissance capsule used primarily by the US Air Force and was designed to be carried by the Voodoo RF-101 reconnaissance aircraft that employed 70 millimeter film. These capsules were capable of capturing images of ground targets up to a resolution of around two and a half centimeters or approximately one inch and could be programmed to capture images at specific intervals or by command, the an asd2 had a number of advanced features for its time. , including a rotating prism that allowed the camera to capture images from multiple angles and a stabilization system that reduced camera shake during flight, enabling higher speed, lower altitude missions.

The first digital imaging system to be used for reconnaissance were the optical components of the advanced synthetic aperture radar system or asars installed on the U2 reconnaissance aircraft in the In the late 1970s, Acer used a large phased antenna to create high-resolution images of the ground below using radar that complemented the radar. It was an imaging system that used a charged pair device or CCD camera to capture visible light images of the terrain being surveyed. This CCD camera operated. in synchronization with the radar system and had a resolution of about 1 meter or 3.3 feet per pixel, a CCD sensor consists of a series of small light-sensitive cells, each cell containing a small capacitor that can store a charge Electricity When light hits the surface of the CCD, it generates an electrical charge within each cell that is proportional to the intensity of the light.

The charge from each cell is then moved along a series of parallel conducting channels called registers using a timing signal. This load moves through the registers in a process called loading. coupled transfer once the load reaches the end of the registers, it is read one row at a time and digitally quantized by an analog to digital converter where it is then processed and stored. CCD sensors would quickly become standard

technology

for military surveillance as they enabled imaging. in a compact, durable and lightweight package; However, despite these advantages, early CCDs were expensive to manufacture and difficult to produce at high cell densities when combined with the limitations of computer hardware of the time, their designs were generally limited to less than one megapixel resolutions. so low In the early 1990s, when 100,000 pixels were found in some systems, a new class of image sensors called active pixel sensors, based primarily on the CMOS manufacturing process, began to permeate the commercial market.

Active pixel sensors employ multiple transistors at each photo site to amplify and move charge using a traditional signal path, making the sensor much more flexible for different applications because of this pixel. Standalone CMOS sensors also use more conventional and less expensive manufacturing techniques already established for semiconductor manufacturing production lines. These sensors also consumed up to 100 times less. power than an equivalent CCD sensor, although these advantages came at the cost of being less sensitive and greater susceptibility to noise in the 1990s, dramatic advances in embedded computing, and the declining costs of memory and computing power They would bring a new era of low-cost digital

technology

.

Imaging Technologies in 2000 CMOS sensors were used in a variety of applications, including low-cost

cameras

, PC

cameras

, and security systems. The demand for higher resolutions and lower noise performance would drive the technology even further, and its proliferation in camera phones would have a profound cultural impact. As technology became intertwined with the advent of social media in 2001, at Lawrence Livermore National Laboratory, the energy department would start a new program aimed at monitoring nuclear proliferation, called the Sonoma Persistent Surveillance program. The core of this initiative was a new class of surveillance technology known as wide-area motion imaging. Wide-area motion imaging takes a completely different approach to traditional ISR technologies by using panoramic optics combined with an extremely dense image sensor. in concept, which would allow finer localized details to be resolved in real time. time purely for the large amount of information captured within the wide optical field.

In 2005, the project now called Constant Hawk was transferred to the US Department of Defense with MIT laboratories conducting the primary research for the program. The first iteration of the Constant Hawk optical sensor was created by combining six 11-megapixel CMOS image sensors that captured only the intensity of visible light and some infrared light without color information with a combined density of 66 megapixels, each frame generating about of 99 megabytes of data when capturing, processing and storing such a large data stream within an airborne system was a challenge in itself, but to truly realize the full potential of Constant Hawk would require real-time image analysis. real.

It would take more than four years of research and testing to develop processing algorithms that would make it possible to extract actionable information in real time. From the large image data set at an altitude of 20,000 feet, the Constant Hawk was designed to survey a circular area on the ground with a radius of approximately 96 kilometers or 60 miles, covering a total area of ​​more than 28,500 square kilometers or about 11,000 square miles at this altitude, each pixel can be resolved down to just one-third of a meter or about 14 inches to be able to conduct surveillance over such a large area in real time, the MIT team devised an approach that focused in detecting changes in the integrated image processors. would align and match every captured frame of the area and only store snapshots of regions where changes occurred that region now made it possible to access any event at any time that occurred within the range of the system and flight duration emissions.

Real-time investigation of a chain of events over a large area was now possible on an ISR mission in 2006 Constant Hawk became the first wide-area moving imagery platform to be deployed as part of the Army's quick reaction capability to help combat enemy ambushes and improvised explosive devices in Iraq installed aboard the modest, short 360 aircraft. . 300 operators were able to employ the system to track the sudden appearance of parked aircraft vehicles or vehicle track and trace activity in key areas that would indicate a potential threat, these changes in the detector would often trigger further investigations or, if the event had already occurred , a search through the archives to determine exactly when and how things changed this ability to track the Falcon's continued effectiveness in Iraq would lead to its deployment to Afghanistan in 2009, although now aboard a Liberty MC-12W aircraft because the core Of the Hawk's consistent capabilities came from its software, it was relatively easy to reconfigure and adapt, especially when CMOS image sensor technology began to advance rapidly due to the emergence of the smartphone in 2009, BAE Systems would add night vision capabilities and It would increase the sensor density to 96 megapixels, all while reducing the size of the original sensor package from 680 kilograms or 1,500 pounds in 2013.

Full-color image processing capabilities would be added at this point. Hawk was able to generate nearly 15 terabytes of reconnaissance data per hour. The system was so successful that the Marine Corps would adopt elements of the program to create its own system called Angel Fire and even a derivative system called Kestrel would be installed on a stationary airstat in Afghanistan that would function as a large area aerial Sentry, as it Constant Hawk was seeing its first deployment several years ago. Other systems were being developed that targeted more specific ISR functions; However, one system in particular would create a new class of aerial surveillance previously thought impossible called The Argus is or the Real-Time Autonomous Ground Ubiquitous Surveillance Imaging System that this DARPA projecthired BAE.

The systems aimed to image an area at such a high level of detail and frame rate that it could collect living data patterns that specifically tracked individuals within the sensor field. What makes the concept so sinisterly powerful is the 1.8-gigapixel image sensor at its core derived from smartphone camera technology. The sensors made with four lenses and 368 5 megapixel CMOS image sensors compared to the constant Hawk Argus are specifically designed to cover a much narrower field of view of around 16 kilometers or approximately 10 miles when combined with such a sensor. dense that each pixel can resolve with just two millimeters of detail, allowing you to identify faces, clothing, defining features such as hairstyles, body shapes and tattoos, and even read text as with Constant Hawk.

A large part of Argus's development revolved around processing the immense amounts of data produced operating at a base capture rate of 15 frames per second, the system generates almost 21 terabytes of color images every second because Argus is designed specifically to track a constant processing system derived from Project Hawk called persistics. Similarly, it only captures and processes changes that occur within the viewing area while building a chronological database of images. However, Argus takes this capability even further with the ability to actively track up to 65 different objects simply from optically defining signatures and builds a searchable object motion timeline because this tracking can be done backwards.

Now the system becomes a powerful tool for forensic investigators and intelligence analysis of human behavior patterns. Argus completed its initial round of flight testing in early 2010 aboard a Blackhawk helicopter and by 2014 had achieved operational capability with the adoption of variants of its core design. Unlike other scaled-down wide-area moving imaging platforms today, Argus and its derived variants have been used operationally around the world aboard large and small UAV aerostats and manned aircraft, although the extent of its use and advances made The system has been kept out of public access since 2016. It is speculated that with current computing power, the ability to generate detailed 3D models for areas of interest from a combination of imagery and complementary lidar instruments is now possible and can be part of Argus' current capacity as of February 2023, the top tier of consumer CMOS image sensor manufacturers produce sensors in the 120-megapixel range, with 200-megapixel sensors expected on the horizon.

These sensors are also supported by the powerful image processing capabilities of current ARM-based processors along with a dense yet efficient low cost. Current Storage Technologies: The capabilities of the initial constant Hawk system could easily be matched for a small fraction of the cost of that program in a much smaller, resilient and flexible package. Therefore, one can speculate that a system not disclosed to the public would conceptually offer disturbingly unprecedented levels. resolution and capability, particularly if the technology migrates from military to civilian law enforcement. Beyond civil liberty concerns, the simple fact that a small but effective wide-area moving image system can now be created easily and at low cost from consumer hardware poses a new threat because of the potential of The massive deployment of such devices by foreign adversaries who could evade defensive measures simply by the sheer number of slow-moving aerial vehicle boards whose intent can be easily disguised.

One of the interesting results of wide area moving image programs has been the expansion of the technology. In the civilian world with the help of data analysis techniques such as machine learning and computer vision, wide area motion imaging can now be used to detect and track moving objects over large geographic areas, providing insights and intelligence. critiques a variety of industries with data science now. Shaping Your Future Have you ever wanted to build a solid foundation for understanding how massive sets of data can be used effectively to understand the world around us? That's where shiny.org comes into play. shiny.org is my go-to tool for diving head first into learning a new concept.

It's a website and app built on the principle of active problem solving because to really learn something you need more than just seeing it, you have to experience it. Brilliance is constantly developing its courses to offer the most visual hands-on approach possible. To make mastering the key concepts behind today's technology effective and engaging, I have personally found their data science and analytics courses to be extremely valuable, allowing you to build, examine, and self-discover the key concepts of data analytics. through the eyes of statistical thinking. Using interactive exercises with brilliants you learn in depth and at your own pace it is not about memorizing or regurgitating data you simply choose a course that interests you and start if you feel stuck or made a mistake there is always an explanation available to help you through the learning process , try everything Bright has to offer free for a full 30 days and start learning Stem today.

Visit shiny.org. Sweep forward New Mind or click the link in the description. The first 200 of you will get 20 off Brilliant's annual premium. subscription