A new way to work with three.js, 3D models and rendering on the web.

ThreePipe — Github — Examples — API Reference — WebGi

ThreePipe is a modern 3D framework built on top of three.js, written in TypeScript, designed to make creating high-quality, modular, and extensible 3D experiences on the web simple and enjoyable.

Key features include:

• Simple, intuitive API for creating 3D model viewers/configurators/editors on web pages, with many built-in presets for common workflows and use-cases.
• Companion editor to create, edit and configure 3D scenes in the browser.
• Modular architecture that allows you to easily extend the viewer, scene objects, materials, shaders, rendering, post-processing and serialization with custom functionality.
• Plugin system along with a rich of built-in plugins that allows you to easily add new features to the viewer.
• uiconfig compatibility to automatically generate configuration UIs in the browser.
• Modular rendering pipeline with built-in deferred rendering, post-processing, RGBM HDR rendering, etc.
• Material extension framework to modify/inject/build custom shader code into existing materials at runtime from plugins.
• Extendable asset import, export and management pipeline with built-in support for gltf, glb, obj+mtl, fbx, materials(pmat/bmat), json, zip, png, jpeg, svg, webp, ktx2, ply, 3dm and many more.
• Automatic serialization of all viewer and plugin settings in GLB(with custom extensions) and JSON formats.
• Built-in undo/redo support for user actions.
• Automatic disposal of all three.js resources with built-in reference management.
• Realtime Realistic Rendering with screen-space post-processing effects from webgi.
• Animation system(and UI) to create state, keyframe-based animations for any object, material, or viewer property with global timeline.

Checkout the documentation and guides on the threepipe website for more details.

Code samples and demos covering various usecases and test are present in the examples folder.

Try them: https://threepipe.org/examples/

View the source code by pressing the code button on the top left of the example page.

To make changes and run the example, click on the CodePen button on the top right of the source code.

Checkout the full Getting Started Guide on threepipe.org

To create a new project locally

npm create threepipe@latest

And follow the instructions to create a new project.

Get started with pre-ready templates with model viewer and plugins that run locally directly in your browser –

• javascript

• typescript

• javascript + webgi plugins

• typescript + webgi plugins

<canvas id="three-canvas" style="width: 800px; height: 600px;"></canvas>
<script type="module">
  import {ThreeViewer, DepthBufferPlugin} from 'https://unpkg.com/threepipe@latest/dist/index.mjs'

  const viewer = new ThreeViewer({canvas: document.getElementById('three-canvas')})

  // Add some plugins 
  viewer.addPluginSync(new DepthBufferPlugin())
  
  // Load an environment map
  const envPromise = viewer.setEnvironmentMap('https://samples.threepipe.org/minimal/venice_sunset_1k.hdr')
  const modelPromise = viewer.load('https://threejs.org/examples/models/gltf/DamagedHelmet/glTF/DamagedHelmet.gltf', {
    autoCenter: true,
    autoScale: true,
  })

  Promise.all([envPromise, modelPromise]).then(([env, model]) => {
    console.log('Loaded', model, env, viewer)
  })
</script>

Check it in action: https://threepipe.org/examples/#html-js-sample/

Check out the details about the ThreeViewer API and more plugins.

The best way to use the viewer in react is to wrap it in a custom component.

Here is a sample react component in tsx to render a model with an environment map.

import React from 'react'
function ThreeViewerComponent({src, env}: {src: string, env: string}) {
  const canvasRef = React.useRef(null)
  React.useEffect(() => {
    const viewer = new ThreeViewer({canvas: canvasRef.current})

    const envPromise = viewer.setEnvironmentMap(env)
    const modelPromise = viewer.load(src)
    Promise.all([envPromise, modelPromise]).then(([env, model]) => {
      console.log('Loaded', model, env, viewer)
    })
    
    return () => {
      viewer.dispose()
    }
  }, [])
  return (
     <canvas id="three-canvas" style={{width: 800, height: 600}} ref={canvasRef} />
  )
}

Check it in action: https://threepipe.org/examples/#react-tsx-sample/

Other examples in js: https://threepipe.org/examples/#react-js-sample/ and jsx: https://threepipe.org/examples/#react-jsx-sample/

For a more declarative approach using JSX syntax, you can use the @threepipe/plugin-r3f package which provides React Three Fiber integration.

npm install @threepipe/plugin-r3f

Here is a sample React Three Fiber component to render a model with an environment map using declarative JSX syntax.

import React from 'react'
import { createRoot } from 'react-dom/client'
import { ViewerCanvas, Asset, Model } from '@threepipe/plugin-r3f'
import { LoadingScreenPlugin } from 'threepipe'

function App() {
  return (
    <ViewerCanvas
      id="three-canvas"
      style={{width: 800, height: 600}}
      plugins={[LoadingScreenPlugin]}
      onMount={async (viewer) => {
        console.log('Viewer mounted:', viewer)
      }}
    >
      <React.Suspense fallback={<mesh>
        <boxGeometry args={[1, 1, 1]} />
        <meshStandardMaterial color="orange" />
      </mesh>}>
        <Asset 
          url="https://samples.threepipe.org/minimal/venice_sunset_1k.hdr"
          autoSetBackground={true}
        />
        <Asset 
          url="https://threejs.org/examples/models/gltf/DamagedHelmet/glTF/DamagedHelmet.gltf"
          autoCenter={true}
          autoScale={true}
        />
      </React.Suspense>
    </ViewerCanvas>
  )
}

createRoot(document.getElementById('root')).render(<App />)

ViewerCanvas is the wrapper around the r3f Canvas component that initializes the ThreePipe viewer and provides the viewer context to all child components.

Any children added to this component are added to the scene model root.

Check it in action: https://threepipe.org/examples/#r3f-tsx-sample/

A sample vue.js component in js to render a model with an environment map.

const ThreeViewerComponent = {
  setup() {
    const canvasRef = ref(null);

    onMounted(() => {
      const viewer = new ThreeViewer({ canvas: canvasRef.value });

      const envPromise = viewer.setEnvironmentMap('https://samples.threepipe.org/minimal/venice_sunset_1k.hdr');
      const modelPromise = viewer.load('https://threejs.org/examples/models/gltf/DamagedHelmet/glTF/DamagedHelmet.gltf');

      Promise.all([envPromise, modelPromise]).then(([env, model]) => {
        console.log('Loaded', model, env, viewer)
      })

      onBeforeUnmount(() => {
        viewer.dispose();
      });
    });

    return { canvasRef };
  },
};

Check it in action: https://threepipe.org/examples/#vue-html-sample/

Another example with Vue SFC(Single file component): https://threepipe.org/examples/#vue-sfc-sample/

A sample svelte component in js to render a model with an environment map.

<script>
    import {onDestroy, onMount} from 'svelte';
    import {ThreeViewer} from 'threepipe'; 

    let canvasRef;
    let viewer;
    onMount(() => {
        viewer = new ThreeViewer({canvas: canvasRef});

        const envPromise = viewer.setEnvironmentMap('https://samples.threepipe.org/minimal/venice_sunset_1k.hdr');
        const modelPromise = viewer.load('https://threejs.org/examples/models/gltf/DamagedHelmet/glTF/DamagedHelmet.gltf');

        Promise.all([envPromise, modelPromise]).then(([env, model]) => {
          console.log('Loaded', model, env, viewer)
        })
    });
    onDestroy(() => viewer.dispose())
</script>

<canvas bind:this={canvasRef} id="three-canvas" style="width: 800px; height: 600px"></canvas>

Check it in action: https://threepipe.org/examples/#svelte-sample/

For Svelte 5, simply initialize canvasRef to $state() –

let canvasRef = $state();

npm install threepipe

First, create a canvas element in your HTML page:

<canvas id="three-canvas" style="width: 800px; height: 600px;"></canvas>

Then, import the viewer and create a new instance:

import {ThreeViewer, IObject3D} from 'threepipe'

// Create a viewer
const viewer = new ThreeViewer({canvas: document.getElementById('three-canvas') as HTMLCanvasElement})

// Load an environment map
await viewer.setEnvironmentMap('https://samples.threepipe.org/minimal/venice_sunset_1k.hdr')

// Load a model
const result = await viewer.load<IObject3D>('https://threejs.org/examples/models/gltf/DamagedHelmet/glTF/DamagedHelmet.gltf', {
    autoCenter: true,
    autoScale: true,
})

That’s it! You should now see a 3D model on your page.

The 3D model can be opened in the editor to view and edit the scene settings, objects, materials, lights, cameras, post-processing, etc. and exported as a GLB file. All settings are automatically serialized and saved in the GLB file, which can be loaded into the viewer. Any plugins used in the editor can be added to the viewer to add the same functionality. The plugin data is automatically loaded(if the plugin is added) when the model is added to the scene.

The viewer initializes with a Scene, Camera, Camera controls(Orbit Controls), several importers, exporters and a default rendering pipeline. Additional functionality can be added with plugins.

Check out the glTF Load example to see it in action or to check the JS equivalent code: https://threepipe.org/examples/#gltf-load/

Check out the Plugins section to learn how to add additional functionality to the viewer.

The core framework(src, dist, examples folders) and any plugins without a separate license are under the Free Apache 2.0 license.

Some plugins(in the plugins folder) might have different licenses. Check the individual plugin documentation and the source folder/files for more details.

The project is in beta stage and under active development. Many features will be added but the core API will not change significantly in future releases.

• ThreePipe
  • Examples
  • Table of Contents
  • Getting Started
    • HTML/JS Quickstart (CDN)
    • React
    • React Three Fiber (R3F)
    • Vue.js
    • Svelte
    • NPM/YARN Package
      • Installation
      • Loading a 3D Model
  • License
  • Status
  • Documentation (API Reference)
  • Contributing
• Features
  • File Formats
  • Loading files
  • Exporting files
  • Render Pipeline
  • Material Extension
  • UI Configuration
  • Serialization
  • Plugin System
  • Timeline
• Viewer API
  • ThreeViewer
  • RenderManager
  • RootScene
  • ICamera
  • AssetManager
    • AssetImporter
    • AssetExporter
    • MaterialManager
  • Other classes and interfaces
• Plugins
  • TonemapPlugin – Add tonemap to the final screen pass
  • DropzonePlugin – Drag and drop local files to import and load
  • ProgressivePlugin – Post-render pass to blend the last frame with the current frame
  • SSAAPlugin – Add Super Sample Anti-Aliasing by applying jitter to the camera.
  • DepthBufferPlugin – Pre-rendering of depth buffer
  • NormalBufferPlugin – Pre-rendering of normal buffer (deferred shading)
  • GBufferPlugin – Pre-rendering of depth-normal and flags buffers in a single pass (deferred shading)
  • SSAOPlugin – Add SSAO(Screen Space Ambient Occlusion) for physical materials.
  • CanvasSnapshotPlugin – Add support for taking snapshots of the canvas with anti-aliasing and other options
  • PickingPlugin – Adds support for selecting objects in the viewer with user interactions and selection widgets
  • AssetExporterPlugin – Provides options and methods to export the scene/object GLB or Viewer Configuration JSON
  • LoadingScreenPlugin – Shows a configurable loading screen overlay over the canvas
  • FullScreenPlugin – Adds support for moving the canvas or the container fullscreen mode in browsers
  • InteractionPromptPlugin – Adds an animated hand icon over canvas to prompt the user to interact
  • TransformControlsPlugin – Adds support for moving, rotating and scaling objects in the viewer with interactive widgets
  • ContactShadowGroundPlugin – Adds a ground plane at runtime with contact shadows
  • GLTFAnimationPlugin – Add support for playing and seeking glTF animations
  • PopmotionPlugin – Integrates with popmotion.io library for animation/tweening
  • AnimationObjectPlugin – Create and manage keyframe-based animations for any object, material, or viewer property with timeline controls
  • CameraViewPlugin – Add support for saving, loading, animating, looping between camera views
  • TransformAnimationPlugin – Add support for saving, loading, animating, between object transforms
  • RenderTargetPreviewPlugin – Preview any render target in a UI panel over the canvas
  • GeometryUVPreviewPlugin – Preview UVs of any geometry in a UI panel over the canvas
  • FrameFadePlugin – Post-render pass to smoothly fade to a new rendered frame over time
  • VignettePlugin – Add Vignette effect by patching the final screen pass
  • ChromaticAberrationPlugin – Add Chromatic Aberration effect by patching the final screen pass
  • FilmicGrainPlugin – Add Filmic Grain effect by patching the final screen pass
  • NoiseBumpMaterialPlugin – Sparkle Bump/Noise Bump material extension for PhysicalMaterial
  • CustomBumpMapPlugin – Adds multiple bump map support and bicubic filtering material extension for PhysicalMaterial
  • ClearcoatTintPlugin – Clearcoat Tint material extension for PhysicalMaterial
  • FragmentClippingExtensionPlugin – Fragment/SDF Clipping material extension for PhysicalMaterial
  • ParallaxMappingPlugin – Relief Parallax Bump Mapping extension for PhysicalMaterial
  • HDRiGroundPlugin – Add support for ground projected hdri/skybox to the webgl background shader.
  • VirtualCamerasPlugin – Add support for rendering virtual cameras before the main one every frame.
  • EditorViewWidgetPlugin – Adds an interactive ViewHelper/AxisHelper that syncs with the main camera.
  • Object3DWidgetsPlugin – Automatically create light and camera helpers/gizmos when they are added to the scene.
  • Object3DGeneratorPlugin – Provides an API and UI to create scene objects like lights, cameras, meshes, etc.
  • DeviceOrientationControlsPlugin – Adds a controlsMode to the mainCamera for device orientation controls(gyroscope rotation control).
  • PointerLockControlsPlugin – Adds a controlsMode to the mainCamera for pointer lock controls.
  • ThreeFirstPersonControlsPlugin – Adds a controlsMode to the mainCamera for first person controls from threejs.
  • GLTFKHRMaterialVariantsPlugin – Support using for variants from KHR_materials_variants extension in glTF models.
  • Rhino3dmLoadPlugin – Add support for loading .3dm files (Rhino 3D)
  • PLYLoadPlugin – Add support for loading .ply files
  • STLLoadPlugin – Add support for loading .stl files (STL)
  • KTX2LoadPlugin – Add support for loading .ktx2 files (KTX2 – GPU Compressed Textures)
  • KTXLoadPlugin – Add support for loading .ktx files (Note – use ktx2)
  • USDZLoadPlugin – Add partial support for loading .usdz, .usda files (USDZ)
  • GLTFMeshOptDecodePlugin – Decode glTF files with EXT_meshopt_compression extension.
  • SimplifyModifierPlugin – Boilerplate for plugin to simplify geometries
  • MeshOptSimplifyModifierPlugin – Simplify geometries using meshoptimizer library
  • UndoManagerPlugin – Adds support for undo/redo operations in the viewer. Used by other plugins to manage undo history.
  • ObjectConstraintsPlugin – Add support for constraints between objects like follow path, look at, position/rotation/scale locking, etc.
• Packages
  • @threepipe/webgi-plugins – Web Global Illumination – Realistic rendering plugin pack (SSR, SSRTAO, HDR Bloom, TAA, Depth of Field, SSGI, etc.)
  • @threepipe/plugin-tweakpane Tweakpane UI Plugin
  • @threepipe/plugin-blueprintjs BlueprintJs UI Plugin
  • @threepipe/plugin-tweakpane-editor – Editor Plugin using Tweakpane for plugin UI
  • @threepipe/plugin-configurator – Provides MaterialConfiguratorPlugin and SwitchNodePlugin to allow users to select variations
  • @threepipe/plugin-gltf-transform – Plugin to transform glTF models (draco compression)
  • @threepipe/plugins-extra-importers – Plugin for loading more file types supported by loaders in three.js
  • @threepipe/plugin-blend-importer – Add support for loading .blend file. (Partial/WIP) (Blender)
  • @threepipe/plugin-geometry-generator – Generate parametric geometry types that can be re-generated from UI/API
  • @threepipe/plugin-gaussian-splatting – 3D Gaussian Splatting plugin for loading and rendering splat files
  • @threepipe/plugin-network – Network/Cloud related plugin implementations for Threepipe
  • @threepipe/plugin-svg-renderer – Add support for exporting a 3d scene as SVG using three-svg-renderer
  • @threepipe/plugin-3d-tiles-renderer – Plugins for 3d-tiles-renderer, b3dm, i3dm, cmpt, pnts, dzi, slippy maps importers
  • @threepipe/plugin-path-tracing – Plugins for path-tracing. Using three-gpu-pathtracer
  • @threepipe/plugin-assimpjs – Plugin and helpers to load and use assimpjs (with fbx, other exporters) in the browser
  • @threepipe/plugin-timeline-ui – A timeline UI component and plugin to manage global viewer timeline and animations
  • @threepipe/plugin-r3f – React Three Fiber integration. Provides React components for declarative 3D experiences with ThreePipe viewer context
  • @threepipe/plugin-troika-text – troika-three-text integration plugin that provides high performance SDF Text

Check the list of all functions, classes and types in the API Reference Docs.

Contributions to ThreePipe are welcome and encouraged! Feel free to open issues and pull requests on the GitHub repository.

The DNSPod User API OR DNSPod中文文档 is restricted to individual users, making it easier and more flexible for users to manage their own domain names and records.

https://www.dnspod.com China Hong Kong

https://www.dnspod.cn China Shandong Province

Need to cooperate with Curl extension

Use Composer to install the library. Of course, You can go to Packagist to view.

    $ composer require yakeing/dnspod_api

• Sample：

    $uid = 12345;
    $token = X12345;
    $DP = new Dnspod($uid, $token);

• Sample：

    $domain = 'example.com';
    $value = array(
        '255.255.255.1',
        '255.255.255.2',
        '255.255.255.3',
        );
    $name = 'www';
    $type = 'A';
    $DP->Records($domain, $value, $name, $type, true);

• Sample：

    $domain = 'example.com';
    $DP->copyArecord($domain);

• Sample：

    $copyDomain = 'google.com';
    $toDomain = 'example.com';
    echo $DP->getDomainInfo($copyDomain, $toDomain);

• Sample：

    $domain = 'example.com';
    echo $DP->getRecordList($domain);

• Sample：

    $job_id = 'j12345';
    echo $DP->getBatchDetail($job_id);

• Sample：

    $domain = 'example.com';
    $name = 'www';
    $value = '255.255.255.0';
    $type = 'A';
    echo $DP->addRecord($domain, $name, $value, $type);

• Sample：

    $domain_id = '12345';
    $record[0] = array('name'=>'WWW', 'type'=>'A', 'value'='255.255.255.0', 'mx'=>1);
    echo $DP->batchAddRecord($domain_id, $record);

• Sample：

    $domain = 'example.com';
    $record_id = 'E12345';
    $name = 'WWW2';
    $value = '255.255.255.0';
    $type = 'A';
    $mx = 1;
    echo $DP->recordModify($domain, $record_id, $name, $value, $type, $mx);

• Sample：

    $domain = 'example.com';
    $record_id = 'E12345';
    echo $DP->recordRemove($domain, $record_id);

• Sample：

    //Get the API version number
    echo $DP->getVersion();

    //Get the level allowed line
    $domain = 'example.com';
    echo $DP->getRecordLine($domain);

    //Get a list of domain names
    echo $DP->getDomainList();

    //Construct a new record table
    $name = 'example.com';
    $type = 'A';
    $value = '255.255.255.0';
    $DP->newRecords($name, $type, $value);

If you’ve got value from any of the content which I have created, then I would very much appreciate your support by payment donate.

weibo: yakeing

twitter: yakeing

This is a first version of a gRPC service and protobuf definition for serving NAIP metadata that tries to be STAC compliant.

Requirements:

• aws cli tool
• AWS s3 requester pays authorization in ~/.aws/credentials (aws configure to setup)
• ogr2ogr with postgres extensions
• docker + docker-compose
• virtualenv

There are two different ways to test the gRPC service. One is with a postgis docker container and the other is with a docker-compose db+service. Both require this initial set of commands.

Initial commands will download a gig or more of shapefiles to your local machine and prepare a database:

git clone git@github.com:geo-grpc/naip-stac-grpc.git
cd naip-stac-grpc
virtualenv venv
source venv/bin/activate
./download_shapefiles.sh
./naip_import_aws.sh

Test version 1; test with the local docker database:

pip3 install -r requirements.txt
python3 setup.py install
python3 service.py
python3 test_client.py

Test version 2; test with docker-compose initialized with a pg_dump file:

export PG_DB_DATA=naip_visual_db-$(date +%Y-%m-%d)
# dump postgres table to file
docker exec naip-metadata-postgis pg_dump -U user -Fc \
  -t naip_visual testdb > ./$PG_DB_DATA
docker stop naip-metadata-postgis
docker-compose up --build -d
# wait for postgres db to initialize. you could omit the `-d` and 
# watch for the initialization completion and execute the rest of 
# the command from another window
sleep 15
docker exec -i naip-stac-grpc-db-c pg_restore -C --clean --no-acl --no-owner \
  -U user -d testdb < ./$PG_DB_DATA
pip3 install -r requirements.txt
python3 test_client.py

The definitions for a stac item response are in protos/epl/protobuf/stac_item_result.proto. It is copied from the protocol buffer for stac defined here:

• https://github.com/geo-grpc/protobuf

There are a couple of distinctions from the STAC definitions.

• there isn’t a properties container on the item result object. It could be added, but for the purposes of the demo it made it more difficult.
• there isn’t a bands array on the item result object.

Protobuf definitions have fields that are indexed by field numbers. As we want people to extend STAC for there own purposes the field numbers 201 to 500 are available for custom definitions. The field numbers from 1 to 200 and from 501 to max are reserved for STAC definitions. More keys could be released as needed.

There are two different versions of the proto file format, proto2 and proto3 that are currently in use. For the message response, the stac_item_result.proto is defined for proto2. In protobuf, messages are like structs. They must have a default value even if that value hasn’t been set, and in the name of compactness that value is 0. In proto2, the version
of our proto file for results, there is a method that allows you to check whether a field has been set (this is absent from proto3). That way you can ignore values that are 0, but doesn’t represent the data. If the HasField method returns false, the data should be ignored. HasField, is a poor name, because there is still a field there is data, it’s just the data isn’t set by the creator of the message.

This stac experiment imports a geometry proto definition used in another gRPC project. One of the aspects of this geometry definition is that you can define your aoi geometry using a wkt, wkb, geojson or esrishape. GeoJSON shouldn’t be the only option, especially if a user wants results that are more compact. By default this project returns wkb for compactness, though it can accept wkt or wkb as an input.

install requirements:

pip3 install -r requirements.txt

The repo contains compiled python files generated from the included proto file definitions. If you choose to make changes to the proto files you’ll need to compile the proto files to python code. The generated code is in the epl/grpc and the epl/protobuf directories.

python3 -mgrpc_tools.protoc -I=./protos --python_out=./ \
    ./protos/epl/protobuf/geometry_operators.proto \ 
    ./protos/epl/protobuf/stac.proto
python3 -mgrpc_tools.protoc -I=./protos --grpc_python_out=./ \
    ./protos/epl/grpc/naip_stac.proto

after compiling the proto files included above , install the packages:

python3 setup.py install

You need an AWS account with s3 requester pays read access. ogr2ogr with Postgres plugin is required for writing data to DB. Docker is required for running the DB.

AWS Setup:
~/.aws/credentials with aws_access_key_id and aws_secret_access_key. NAIP bucket is requester pays.

GDAL + ogr2ogr + postgresql:

brew install gdal2 --with-postgresql

To collect all the data data from the AWS NAIP shapefiles (a gig or more) you’ll need to execute the included bash script, naip_import_aws.sh.

Once the naip_import_aws.sh script is finished and you have the database up and running you can run the tests. From within the repo directory you can call pytest to run all tests. There will be some warnings, from psycopg2 but beyond that all tests should pass.

pytest

To test the service you can open a terminal and run python3 service.py and from another terminal run python3 test_client.py, or run the jupyter notebook from the repo.

STAC is described in further detail here:

• https://github.com/radiantearth/stac-spec
• https://github.com/radiantearth/stac-spec/blob/master/item-spec/item-spec.md
• https://github.com/radiantearth/stac-spec/tree/master/extensions/eo
• https://medium.com/radiant-earth-insights/the-state-of-stac-talk-and-sprint-3-recap-cd8eda3b8bdb

gRPC services, protobuf binary, and the proto files that define them can be used separately, but they were designed to be used together for microservices communication. They are part of an open source intiaitive from Google. They’re based off of Google’s own internal RPC framework, Stubby. More info can be found here:

• https://grpc.io/
• https://grpc.io/docs/quickstart/python.html
• https://developers.google.com/protocol-buffers/

NAIP data:
AWS and ESRI teamed up to provide a bucket on s3 that is requester pays. More information here:

• https://registry.opendata.aws/naip/

Left to right: Illustrator, Browser, After Effects

# In any valid CEP extension folder:
git clone https://github.com/battleaxedotco/brutalism-api.git

NOTE: This panel is meant to be an example to use alongside your own. It does not require npm run serve and is already in Production context to allow you to run your own alongside it in Developer context (localhost:8080).

• Vue as JS framework
• Hot Reloading (panel updates instantly on every save during npm run serve)
• Vue-CLI-3 as Node.js tooling (webkit)
• bombino as dev tooling and workflow
• Paginations via Vue Router
• Full typescript support for any app with pravdomil’s Adobe types (same as writing .jsx scripts but access to host DOM as autocomplete while typing)
• 🔥 Battleaxe’s own brutalism component library and design 🔥
• Various utility functions and components for ease of panel development
• ✨ All color themes of any host app automatically handled by starlette library ✨

This panel comes with various commands baked in (see details here):

• npm run help – A full list of the commands available and descriptions.
• npm run serve – Start the development server and edit your panel.
• npm run build – Create and compile to production context.
• npm run switch – Reports whether in developer or production context and can switch automatically.
• npm run update – Reports current version of panel in manifest and prompts to update Major, Minor, or Micro.
• npm run register – Reports the current user data (if any) and prompts to save new info to be used in certificates.
• npm run sign – Automatically stages and signs the extension, placing it in a ./archive directory within the current directory.

You can automate this by using npm run switch. In case you need to do it manually:

• Ensure index-dev.html is uncommented in CSXS/manifest.xml

• Run npm run serve in the terminal at the project root
• Launch host application and find in Window > Extensions

Panel now updates in real time and recompiles every time you save in VSCode

• Ensure dist/index.html is uncommented in CSXS/manifest.xml

• Run npm run build in the terminal at the project root
• Launch host application and find in Window > Extensions

Panel is now ready to sign and certify or be used on any client

1. Ensure these lines in ./vue.config.js are commented out:

configureWebpack: {
  // target: "node-webkit",
  // node: false
},

2. Restart your localhost server with npm run serve

3. Launch https://localhost:8080 in a Chrome window

NOTE: The panel cannot interact with the host app while in browser. Script loading and style is given a fallback (second parameter for returned data in evalScript, uses default or defined app/theme/gradient props for Panel component to determine style) while in browser.

• You can easily define script and utility script location to be anywhere in your directory via Panel script-path and utils props

• You can use the ./.certignore file (especially with src included) to manually ignore any specific files or folders during staging

• If in Adobe context and not browser, make sure your target and node lines in ./vue.config.js are not commented out.

• Ensure that starlette is version 1.0.2 or greater.
• Due to Issues #265 and #266 for Animate, <Menus> is automatically given a Switch Theme option for Animate only. The user will need to manually trigger Switch Theme if the host theme is set to dark or light, but all future instances of the panel launch will remain in this theme.

• Scripting files are not a part of hot-reloading and are loaded into memory at your extension’s mounted lifecycle. You will need to Refresh panel in a menu for them to be updated.
• Adding or reorganizing components may cause hot reloading to fail. Many times you will be warned of this in CEF debug‘s console, fix this by hitting ^C in your active terminal to Terminate batch job, then run npm run serve once more and refresh the panel.

• Must run npm run serve and have the App running at: -Local / -Network message in your terminal
• If you launched the app before running npm run serve, click the localhost URL inside the panel’s error message

• Check your CEF client via localhost:#### for an error thrown in your code which breaks the panel’s rendering
• If in Production context and receiving 404 errors in your index.html, ensure your dist/index.html‘s script tags have src attributes such as src=./filepath or src=filepath instead of src=/filepath (leading slash is default but will break, should be fixed via vue.config.js)

• Including hidden files and repositories in your ZXP/ZIP will cause a misleading error message. Be sure to delete hidden items such as node_modules/, .git/, and any other hidden files/folders prior to your sign/certify if not including these in your ./.certignore file.

• esp32 or S2 / S3 / C3 MCU versions in branch develop
• Espressif IDF framework >= 4.2 (4.3 -> 4.4 ideally to support latest S3)
• An epaper display (see Wiki for supported models)
• If you want to have a Web-Service to deliver the image we built our own one. Just head to CALE.es and make an account. This video of UsefulElectronics will help you to get started.

If you are using raw parallel Eink displays in your project we highly recommend trying this new component FastEPD

And our best selling Tindie product the epdiy v7 epaper controller.

ESP32C3 /S3 also works as a target. Please check also config-examples/C3-riscv-spi where is a PIN configuration that is prove to be working. Then just select one of the SPI examples, and do a: idf.py set-target esp32c3

idf.py –preview set-target esp32s3 (Only v4.4 since tried this only with beta3)

Cale-idf is the official ESP-IDF firmware of our Web-Service CALE.es and also the repository where the development of CalEPD epaper component takes place. The main class extends Adafruit GFX so this library has full geometric functions and also fonts including German/Spanish/French special characters support.

In the repository cale-platformio you can have a quick start skeleton to use CalEPD and Adafruit-GFX components, along with optional FocalTech touch I2C. Please be aware that there are some corrections to do by hand until we figure out what is the best way to do it. Read those in the WiKi and please give a ★ to the cale-platformio repository if you find it useful

• We are working in a interesting new PCB design to make a smart switch using this component. If you are interested please check our repository Bistable-smart-Switch and don’t be shy, give it a ★ if you like it.

• A full pfleged version that supports WiFi provisioning using ESP-Rainmaker app is updated on the branch feature/50-idf-v5-rainmaker Note: It needs an external submodule component so don’t forget to run:

  git submodule update –init –recursive

For more news and announcements please check the Wiki section

Please note that parallel driver epdiy is not anymore a requirement and after last update epdiy V6 is not part of this repository, only linked as a git submodule. So in case you want to use our experimental implementation in C++, please pull the git submodules:

git submodule update --init --recursive

Also please notice that if you need to exclude any of the components, like for example epdiy or any other, the fastest and most straigh-forward way is to open the CMakeLists of that component and add as the first line:

return()

That will make this component not to get in the build process. If you are not using EPDiy to drive your epapers, this step is not needed. If you are, please go to: CalEPD/CMakeLists.txt

And enable epdiy in the REQUIRE section and the related classes:

# Uncomment for parallel epapers:
"epdParallel.cpp"
"models/parallel/ED047TC1.cpp"
"models/parallel/ED047TC1touch.cpp"
"models/parallel/ED060SC4.cpp"
# Add more if you need to copying one of the existing, since not all eink sizes are supported

CalEPD has also support for FocalTech and L58 I2C touch panels used in Lilygo parallel epaper EPD047, enabling you to make simple UX interfaces using small epaper displays. This is optional and can be enabled only when the Firmware requires touch. Please check the Wiki for latest news and to see what displays are supported. The Wiki is the perfect place to make updates that are not branch dependant so our documentation efforts will be focused there. CalEPD supports currently the most popular epaper sizes and four color models (4.2, 5.83, 7.5 and 12.48 inches).

• Use develop to try the latest features. Only after days or even weeks of testing, it will be merged in master, and eventually land in a new CalEPD epaper component release
• If you are interested in LVGL / UX please check our project lv_port_esp32-epaper. In this experimental LVGL esp32 fork we are exploring the possibility to make UX in fast paralell displays.

Parallel epapers need to have an EPDiy board or a Lilygo T5-4.7 inches epaper.

Please do not Fork this repository to bookmark it. For that use the ★ Star button. Acceptable forks fall in this three categories:

1. You found a bug and want to suggest a merge request. Then Fork it!
2. You will contribute adding a new epaper model that does not exist or add a new functionality to an existing one.
3. You will use Cale-idf as a base to create something new. But in that case it would be better to fork the components.

This advice is because we don’t like having copies of the whole repository without any reason. But it does not interfere in any way with the Apache License that clearly states that you might reproduce and distribute a copy of this component provided you agree with the terms mentioned there.

If your epaper model is not there just open an Issue and send us one epaper with the SPI interface. If we can make a working implementation and new C++ class then you can use it in your Firmware and we keep the eink as a payment for our effort. If we fail and cannot make a working implementation then it comes back to you at no cost. Also following existing classes you can do it yourself. Just check on the pull requests to see how other developers did to add their epapers!

CALE does only 3 things at the moment and is very easy to set up:

1. It connects to cale.es and downloads a Screen bitmap.
2. In “Streaming mode” it pushes the pixels to Adafruit GFX buffer and at the end renders it in your Epaper.
3. It goes to sleep the amount of minutes you define in the ESP-IDF menuconfig

It wakes up after this deepsleep and goes back to point 1 making it an ideal Firmware if you want to refresh an Events calendar or weather Forecast display. It does not need to be tied to our CALE service. You can use your own full url to your bitmap image. We just recommend to use CALE.es since you can easily connect it to external APIs and have a living epaper. Optionally you can do the same, but with a JPG, using our www-jpg-render example. Please note that in many cases you will require an ESP32-Wrover or similar with PSRAM.

Different cpp examples:

• cale.cpp Main example to use with monochrome or 3 color epapers from Goodisplay/Waveshare
• cale-grayscale.cpp Example only to use with PlasticLogic epapers, serves as footprint to do it with other models
• cale-sensor.cpp Same as cale.cpp but it has a sensor interrupt when a GPIO goes HIGH (Rising) this triggers a new image request
• cale-7-color.cpp Example to retrieve an 4 bits image and send it with up to 7 colors the 5.65 Acep epaper

Best settings on CALE.es website that we found to display color photos with cale-7-color is to use Dither mode: 3×3 and Bits per pixel: 24. This is downgraded to 4bpp using dithering but that’s really ok since 16 colors are more than the epaper supports. It’s not a great photo quality but this epapers where designed to make labels and supermarket prices, not to display quality color pictures.

ROADMAP

2023.Still adding some Goodisplay epapers. Introduction of setMonoMode (to add 4 gray mode in certain models)
2022.Performance optimization and research in parallel eink drivers
2021.Oct→Dec Testing other projects and small pause (Lot's of other work that are not electronics related...)
2021.Aug→Oct Imaging libraries: Adding JPG support and optimizing processes
2021.Jun→Aug Parallel interaction research: UX on epaper displays
2021.Mar till June Enabling touch support to enable UX design in ESP32
2020.Sep Optimizing instantiation and configuration
2020.Aug Adding color epapers 5.83 and 7.5 inches
2020.Jul Added PlasticLogic as a new brand with 4 wire SPI (uses MISO)

CALE-IDF uses this components:

• CalEPD the epaper component
• Adafruit GFX for ESP-IDF My own fork of Adafruit display library
• EPDiy it’s our own fork of the parallel epaper component EPDiy with only the directory structure to use it as an IDF component

They are at the moment included without git submodules so we can develop fast without updating them all the time. But they are also available to be used as your project ESP-IDF components.

Make sure to set the GPIOs that are connected from the Epaper to your ESP32. Data in in your epaper (DIN) should be connected to MOSI:

And then set the image configuration and deepsleep minutes. Here you can also set the rotation for your Eink display:

Optionally if you use touch, for example with 2.7 inches gdew027w3-T epaper, you should configure also FT6X36 Gpios:

Needs 3.3v, a common GND, SDA, SCL for I2C communication, and a input INT pin that signalizes on Low that there is touch data ready.

CalEPD is an ESP-IDF component to drive epaper displays with ESP32 / ESP32S2 and it’s what is sending the graphics buffer to your epaper behind the scenes. It’s designed to be a light C++ component to have a small memory footprint and run as fast as posible, leaving as much memory as possible for your program. Note that the pixels buffer, takes 1 byte to store 8 pixels on each color, so depending on your epaper size may need external PSRAM. Up to 800 * 480 pixels on a monochrome eink it runs stable and there is still free DRAM for more.

master… -> stable version (ChangeLog moved to Wiki)

refactor/oop -> Making the components base, most actual branch, where new models are added. Only after successfull testing they will be merged in master. Inestable branch do not use on Firmware that you ship to a client.

tft_test -> Original SPI master example from ESP-IDF 4 just refactored as a C++ class. Will be kept for historic reasons

Open the /main/CMakeLists.txt file to select what is the C++ file that will be compiled as your main program. Just uncomment one of the SRCS lines:

idf_component_register(
  # Main CALE 
  #SRCS "cale.cpp" -> CALE Firmware for IDF
   SRCS "demo-fonts.cpp"
  #SRCS "demo-epaper.cpp"
  INCLUDE_DIRS ".")

This configuration will just compile the Fonts demo to test your epaper display. Use cale.cpp if you are looking forward to compile our CALE Firmware. This is the ESP-IDF version of that Eink-calendar esp32-arduino Firmware. In CALE.es there is a web-service that can help you making dynamic Screens for your epapers.

Please check Adafruit page on adding new fonts as a start. In order to add the whole character range, you need to set from -> to ranges after calling font convert. Just go to the /components/Adafruit-GFX/fontconvert directory and run:

./fontconvert /usr/share/fonts/truetype/ubuntu/YOURTTF.ttf 18 32 252 > ../Fonts/YOURTTFNAME.h

Be aware that PROGMEM is not supported here since we are not in the Arduino framework. So just remove it from the generated fonts.

As an example with all characters including German umlauts ( ä ö ü and others like ß) I left Ubuntu_M18pt8b ready in the Fonts directory. Be aware that using the whole character spectrum will also take part of your programs memory.

Not being used at the moment since all test and development happens here. Only when there are new working models they will be pushed as new release in the component repository: CalEPD epaper component is published on version 0.9

ESP-IDF uses relative locations as its submodules URLs (.gitmodules). So they link to GitHub. To update the submodules once you git clone this repository:

git submodule update --init --recursive

to download the submodules (components) for this project. Reminder for myself, in case you update the module library just execute:

# pull all changes for the submodules
git submodule update --remote

If it’s an ESP32:

idf.py set-target esp32

If it’s an ESP32S2:

idf.py set-target esp32s2

Make sure to edit Display configuration in the Kconfig menuoptions:

idf.py menuconfig

CALE configuration is the section to set the bitmap URL (non-ssl for the moment), deepsleep until next refresh, and optional display rotation

And then just build and flash:

idf.py build
idf.py flash monitor

To clean and start again in case you change target (But usually no need to run)

idf.py fullclean

To open the serial monitor only

idf.py monitor

Please note that to exit the monitor in Espressif documentation says Ctrl+] but in Linux this key combination is:

Ctrl+5

In the config-examples folder we left samples of GPIO configurations. For example:

• Wave12I48 has the GPIOs ready to use w/Waveshare socket for ESP32-devKitC
• S2 has a sample GPIO config to be used after idf.py set-target esp32s2 (Only for S2 chips)

If you instantiate display.init(true) it activates verbose debug and also lowers SPI frequency to 50000. Most epapers accept a frequency up to 4 Mhz. We did this on debug to be able to sniff with an ESP32 SPI slave “man in the middle” what commands are sent to the display so we can detect mistakes. Even if you print it, is not the same as to hear on the line, this is the real way to reverse engineer something. Hear what the master is saying in a library that works.

+    uint16_t multiplier = 1000;
+    if (debug_enabled) {
+        frequency = 50;
+        multiplier = 1;
+    }

Due to restrictions in C++ that I’m not so aware about there is a limitation when using big integers in the structs { } So SPI frequency is calculated like:

spi_device_interface_config_t devcfg={
    .mode=0,  //SPI mode 0
    .clock_speed_hz=frequency*multiplier*1000,  // --> Like this being the default 4 Mhz
    .input_delay_ns=0,
    .spics_io_num=CONFIG_EINK_SPI_CS,
    .flags = (SPI_DEVICE_HALFDUPLEX | SPI_DEVICE_3WIRE),
    .queue_size=5
};

Feel free to play with Espressif IDF SPI settings if you know what you are doing 😉

A new breed of supported displays is coming being the first the Wave12I48 12.48″ b/w epaper from Waveshare. This is the first component that support this multi epaper displays abstracting their complexity so you can treat it as a normal 1304×984 single display and use all the Adafruit GFX methods and fonts to render graphics over it. Please note that this big display needs a 160 Kb buffer leaving no DRAM available for anything more on your ESP32. So you can either make a very simple program that renders sensor information, or do everything you want, but adding PSRAM for the GFX buffer. Think about ESP32-WROOVER as a good candidate.

In Buffers for big displays like 800*480 where the size is about 48000 bytes long is necessary to feed the watchdog timer and also make a small delay. I’m doing it this way:

+    // Let CPU breath. Withouth delay watchdog will jump in your neck
+    if (i%8==0) {
+       rtc_wdt_feed();
+       vTaskDelay(pdMS_TO_TICKS(1));
+     }

Again, if you know more about this than me, feel free to suggest a faster way. It’s possible to disable also the watchdogs but of course that is not a good practice to do so.

CALE.es Web-service a Web-Service that prepares BMP & JPG Screens with the right size for your displays

CALE.es Arduino-espressif32 firmware

GxEPD Epaper library GxEPD is to use with Espressif Arduino Framework.

Searches that we do to check how other components evolve and what is the position of this one:

“epaper” esp32.com search

“ESP-IDF epaper” Google search

This is the beginning, and a very raw try, to make CALE compile in the Espressif IOT Development Framework. At the moment to explore how difficult it can be to pass an existing ESP32 Arduino framework project to a ESP-IDF based one and to measure how far we can go compiling this with Espressif’s own dev framework. UPDATE: Saved for historical reasons. After starting this project I heavily adopted ESP-IDF as an IoT framework and toolsuit to build firmares. This become also the start of CalEPD that is our own IDF component to control Epapers with ESP32 / ESP32S2.

GxEPD has been a great resource to start with. For CalEPD component, we mantain same Constants only without the Gx prefix and use the same driver nomenclature as GxEPD library, just in small case. Strange-v for the creation of the FocalTech touch library, that I forked to make the FT6X36-IDF component. Hats off to Jean-Marc Zingg that was the first one to make such a great resource supporting so many Eink displays. Please note that there are no plans to port this to Arduino-framework. This repository was specially made with the purpouse to explore Espressif’s own IoT development framework.

Thanks to all the developers interested to test this. Special mentions for @IoTPanic, Spectre and others that pushed me to improve my C++ skills.

If you like this component and it made your life easier please consider becoming a sponsor where you can donate as little as 2 u$ per month. Just click on: ❤ Sponsor on the top right

We are also launching a new company called Marespa.es that will help EU citizens find an affordable apartment in Spain. With the price of Rent going through the roof in 2024, this might be the moment to make an investment, if you plan to work from the spanish coast. With this project we are also supporting our open source projects.

We are thankful for the support and contributions so far!

• Bistable-smart-switch an ESP32-C3 (or S3) smart switch for your wall
• Invisible computers epaper your Google Calendar in an epaper display, drawn using CalEPD!

Go to my GitHub Page fab3F.github.io or to the EasyFlick Homepage easyflick.us.to and join the Discord Server

Hier bei EasyFlick und auf der EasyFlick Homepage findest du alle möglichen Downloads zu Minecraft Ressourcenpaketen, Wallpapern und anderen Dingen. Viel Spaß!

EasyFlick ist ein Netzwerk mit vielen verschiedenen Möglichkeiten in Richtung Gaming, Coding oder auch mal Dokumentationen mit Tipps und Tricks.

- Selbsterstelle Ressourcenpakete
- Eigene Plugins
- Texturepacks
- und vieles mehr...

Wir haben natürlich auch einen Discord, deswegen kommt gerne auf den Discord. 
Dort gibt es viele nette Menschen und eure Fragen werden hier am schnellsten beantwortet! Außerdem

- einen eigenen Discord Bot
- Zocken mit Freunden
- Minigames wie TicTacToe
- Musikbots
- und natürlich einen schnellen Support!

Auf der Homepage gibt es die ganzen Downloads zu den besagten Projekten. 

Du hast ein Problem mit einem unserer Dienste oder möchtest eine Frage loswerden? Dann komm doch gerne auf den EasyFlick Discord! Dort kannst du im extra dafür angelegten Support-Channel nachfragen. Oder du schreibst eine E-Mail an fab3F@programmer.net

Skill for Amazon echo to enable Alexa to talk to Elasticsearch.

Configured IntentSchema:

ElasticCount Count {emptyTerm|term}

Explanation:

2. Search for term in elasticsearch and count result set

Example:

Alexa? Ask Elastic to count error

is transformed to skill (intent) and variable configuration (slots):

intent=ElasticSearch
slot(term)=error

Note: Data type number can be translated from five to 5 directly.

Amazon provided a nice SDK and a nice way to interact with alexa. After registering your skill to amazon developer console, your endpoint get called with relevant payload. I decided to use a spring boot application handling these requests. Java code is in src, relevant business logic is included in

src/main/java/info/unterstein/alexa/elastic/alexa/ElasticSpeechlet.java

Currently you need to configure the target ElasticSearch cluster within code. This should be changed to be configured during installing this skill to amazon echo, see section Option issues. But, for now, you need to go to

src/main/java/info/unterstein/alexa/elastic/ElasticSpeechlet.java

and do something like:

  // TODO
  public ElasticSpeechlet.java() {
    client = new ElasticSearchClient("your.elastic.url", 9300, "your.cluster.name");
  }

Then you need to package this app and start it somewhere:

mvn clean package
# deploy it somewhere with following command
java -jar elastic-alexa-0.0.1-SNAPSHOT.jar --server.port=19002

Text entered:

speechAssets/IntentSchema.json
speechAssets/SampleUtterances.txt

I needed an http endpoint with valid ssl certificate. You can choose between onprem installation or AWS lamba. I decided to deployed the app directly to my server, proxied behind NGINX using the following configuration:

server {
        listen 443 ssl;
        server_name unterstein.info;

...

        ssl_certificate      /etc/nginx/ssl/unterstein.info.crt;
        ssl_certificate_key  /etc/nginx/ssl/unterstein.info.key;

...

        location /alexa {
                proxy_pass http://127.0.0.1:19002/alexa;
                proxy_set_header Host $host;
                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        }
}

At this point it is possible to enable this skill for all amazon echos, registered to the current amazon account and can be used directly.

https://twitter.com/unterstein/status/832302202702196736

• Alexa Java SDK
• Amazon tutorial
• Good pictured tutorial
• Alexa Developer Console

• Make skill configurable so that each user (amazon account) can configure his own ElasticSearch cluster and credentials during skill installation. Currently all users of this skill would talk to the same ElasticSearch cluster, which is .. not really cool :-/
• => See https://developer.amazon.com/public/solutions/alexa/alexa-skills-kit/docs/linking-an-alexa-user-with-a-user-in-your-system

The Hall is a demo created in Cecropia Solutions to test the WebVR capabilities.

The project has a variety of art items in display. Some custom, some of free access and use.

The main are the Teleport Spots, yellow dots on the floor to jump quickly jump to location of interest using the gaze pointer (green circle).

The object modeling and texture baking was created in Blender, where the 1m unit is used as base size; the models are exported individually, converted to CTM and loaded into the application via Three.js.

A couple of the models were created with Oculus Medium and later decimated with Meshlab.

A couple of Kinetic Sculptures were created directly with Three.js and a bit of Math knowledge.

We created a motorcycle model with Photogrammetry.

Captured human movement to create four animations that occupy one side of a hallway.

Live Demo

• WebVR
• Three.js r84
• Gamepad API
• gamepad.js
• Blender
• Wavefront OBJ file format
• CTM file format
• OpenCTM
• Oculus Medium
• Meshlab

• Archive3D
• Nefertiti
• Female with a Helmet

• freesound.org

• wikipedia.org

Live Demo

The Hall is a demo created in Cecropia Solutions to test the WebVR capabilities.

The project has a variety of art items in display. Some custom, some of free access and use.

The main are the Teleport Spots, yellow dots on the floor to jump quickly jump to location of interest using the gaze pointer (green circle).

The object modeling and texture baking was created in Blender, where the 1m unit is used as base size; the models are exported individually, converted to CTM and loaded into the application via Three.js.

A couple of the models were created with Oculus Medium and later decimated with Meshlab.

A couple of Kinetic Sculptures were created directly with Three.js and a bit of Math knowledge.

We created a motorcycle model with Photogrammetry.

Captured human movement to create four animations that occupy one side of a hallway.

Live Demo

• WebVR
• Three.js r84
• Gamepad API
• gamepad.js
• Blender
• Wavefront OBJ file format
• CTM file format
• OpenCTM
• Oculus Medium
• Meshlab

• Archive3D
• Nefertiti
• Female with a Helmet

• freesound.org

• wikipedia.org

Live Demo

IP_Enrich is a tool that given a IP address, it will query multiple security threat intelligence services and enrich the information of the IP with metadata and all the available information on it.

Note: certain services require adding API keys, like VirusTotal and PassiveTotal (RiskIQ).

• Extract VirusTotal data
• Extract PassiveTotal data
• Extract GeoIP data
• Extract Shodan data
• Outputs in a JSON format
• Outputs a nice printed summary
• Can be imported as a module

The following are a list of features that we aim to incorporate to IP Enrich in the future:

• Implement https://api.riskiq.net/api/ssl/
• Implement https://api.riskiq.net/api/blacklist/

First download the code

git clone https://github.com/stratosphereips/ip_enrich.git
cd ip_enrich
python3 ./ip_enrich.py -h

Second, install shodan python library

python -m pip install -r requirements.txt

Now it is ready to run without any API, but if you have any API (even free) from VirusTotal, RiskIQ (Passive Total), or Shodan, put them in their corresponding files. If not it will still run, but you will miss a lot of extra data. We suggest you register for those services and get a free API.

The credentials files are in your home folder

~/.ip_enrich/shodan_credentials
~/.ip_enrich/vt_credentials
~/.ip_enrich/pt_credentials

The format of the pt_credentials is

RiskIQ_email = <email>
RiskIQ_key = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

IP Enrich can be used directly from the command line as an independent tool. An example is shown below:

./ip_enrich.py -i 1.1.1.1
[+] Getting the VirusTotal data
[+] Processing the VirusTotal data
[+] Getting the reverse DNS data
[+] Getting the PassiveTotal data
[+] Getting the Geolocation data
[+] Getting the Shodan data

IP: 1.1.1.1. Country: AU. AS Org: CLOUDFLARENET. RDNS: one.one.one.one.
GeoIP Data
	Country: Australia (AU)
	RegionName: Queensland QLD
	City: South Brisbane
	Lat: -27.4766
	Lon: 153.0166
	TZ: Australia/Brisbane
	isp: Cloudflare, Inc
	Org: APNIC and Cloudflare DNS Resolver project
	AS: AS13335 Cloudflare, Inc.
VT Resolutions (top 10, sorted by datetime):
	2021-11-02 14:00:51: 0002049.xyz
	2021-11-02 06:33:27: 055353.com
	2021-10-28 00:52:26: 0.0www.breadapp.com
	2021-10-28 00:52:25: 0.0.0www.breadapp.com
	2021-10-22 10:14:54: 01eeda8e7e38183e5676cbabe5b8b11e.19f7f31a1a944816d5f44d89024aff48.h.i.ydscan.net
	2021-10-18 13:55:09: 0-v-0.xyz
	2021-10-15 17:32:42: 0.0token.breadapp.com
	2021-10-15 17:32:41: 0.0.0token.breadapp.com
	2021-10-14 23:20:50: 0000jb.com
	2021-10-12 07:54:09: 0.0stage.breadapp.com
VT URLs (top 10, sorted by datetime):
	2021-11-05 17:33:05: http://www.besthotel360.com:1219/001/puppet.Txt?77142. Positives: 1/93
	2021-11-05 16:40:17: http://shenzhengyunde.com/wp-content/plugins/Citrus. Positives: 9/93
	2021-11-05 15:43:12: http://fabianomeroete.gettrials.com/. Positives: 6/93
	2021-11-05 15:10:52: http://korberpie8p6f.servebeer.com/us.html?2Nf8zJ4oH8vPvwUyzhQhY1mO30thIH7MBanBtDZCBtbkNl979971JntUZqTSO6czexqILCwJ2bfvAVECgtX7aNEeQpjIsWc8FF5K_4_2Nf8zJ4oH8vPvwUyzhQhY1mO30thIH7MBanBtDZCBtbkNl979971JntUZqTSO6czexqILCwJ2bfvAVECgtX7aNEeQpjIsWc8FF5K_4. Positives: 4/93
	2021-11-05 14:22:16: http://www.besthotel360.com:1219/001/puppet.Txt?80044. Positives: 1/93
	2021-11-05 14:02:45: http://1.1.1.1/positron/discovery. Positives: 1/93
	2021-11-05 13:04:53: http://thee.network/. Positives: 12/93
	2021-11-05 10:11:49: http://www.besthotel360.com:1219/001/puppet.Txt?82118. Positives: 1/92
	2021-11-05 08:22:00: http://chetverg.xyz/. Positives: 7/92
	2021-11-05 06:09:04: http://www.besthotel360.com:1219/001/puppet.Txt?97687. Positives: 1/92
VT Detected Communicating Samples (top 10, sorted by datetime):
	2021-11-05 18:11:24: Positives: 0, Total: 0, SHA256: f6390d83e5684b3dd5d4b6db71bfd7573a8eb0edcacf548cfb4715ae74eb0ac6
	2021-11-05 18:06:05: Positives: 0, Total: 0, SHA256: e9cbf160213511551b8af68f47e30a7bbca16ef707abb790ee7673cce1a786a4
	2021-11-05 17:57:04: Positives: 0, Total: 0, SHA256: d713e22c9bab1cc73d79f4ea7131ef8cc6ede776b9192474997c50000706b392
	2021-11-05 17:56:14: Positives: 0, Total: 0, SHA256: d4f684092f42598823dc6f9c1a4cf68576924c1700b5d05ae302d0604bd5e21c
	2021-11-05 17:48:25: Positives: 0, Total: 0, SHA256: c39706d752096fa39c44d7f438477721e6ff2cefec04b76ee88808c897d3a4d9
	2021-11-05 17:39:28: Positives: 0, Total: 0, SHA256: adf74bfffcc53e48b4cf4d89839daeb63a6dfefe06c19298f653b3af8bcff5a3
	2021-11-05 17:33:42: Positives: 0, Total: 0, SHA256: 9f48278ecaff72c29f49eb8daa39d99c45369edaae6326da594af7097737a01c
	2021-11-05 17:26:55: Positives: 0, Total: 0, SHA256: 8f0847c175118ed8b533bb5669a90f51c41905e11ec2e04e96741ab2d75f1ce7
	2021-11-05 17:25:59: Positives: 0, Total: 0, SHA256: 8cf24462e9dfdd1aa558c51ee3d91b0da913caf2929debad1d3559a854fc2e61
	2021-11-05 17:19:06: Positives: 0, Total: 0, SHA256: 7da6df949060f6825614f2a08ae687889f2764391b2e8d0941ed68ce26199cff
VT Detected Downloaded Samples (top 10, sorted by datetime):
	2021-09-20 09:51:51: Positives: 1, Total: 72, SHA256: 2c141c06f7df57f11ef2c62f2a96093484a65df47065b1a475c53784af0e2664
	2021-06-26 17:08:59: Positives: 7, Total: 74, SHA256: 8ad3794b215df1a4eaf1325a90a4357ad93476c9308b4e820e325d50eba50280
	2021-04-15 03:35:40: Positives: 1, Total: 73, SHA256: 337dffc1333f286f559c052c45c97f48ac8136cbf6327c24739f058407f45d7d
	2021-04-08 11:30:25: Positives: 1, Total: 74, SHA256: 72ec27bd0d959a1e6713d96b4e55c5a9b92ac6d1b5b5a4a8d5d1211422fcee57
	2021-03-30 15:12:44: Positives: 11, Total: 74, SHA256: 92e9cf96de35f3f9b86c77ded463a4abb7c394a78ea9c14524996de96c920fe9
	2020-10-18 08:17:53: Positives: 18, Total: 75, SHA256: 5a9007b9bcaf5a0a4685a55c2b477fc2b5072e03f98f3f2a310898b27d63d5f1
	2020-06-09 05:28:01: Positives: 4, Total: 74, SHA256: 54b6ce478977f5242698ab1bac90fe11133d2339d1f24fc9d96649099128cd23
	2020-03-14 06:31:57: Positives: 1, Total: 71, SHA256: 1c6c32f969e7f5d9bd7a3361388643db8955b8d3bf72c5fb73ea1b989702ab3e
	2019-09-18 22:43:06: Positives: 1, Total: 72, SHA256: 9f89814b48fc3249bf67a8a6e4439d97391b10b99f02b3da9e38345be1f1ed3f
	2018-04-16 02:49:06: Positives: 23, Total: 62, SHA256: 0773b94a2e3239eeda0d02f32d8beea116783b48172c116c9b6b382338f8be13
VT Detected Referrer Samples (top 10, sorted by sha):
	fe76c029c702ab5f7f6f26e58d56d7dc5a7419947e4b747ef20433c43b456252: Positives: 0, Total: 53
	f7b72d219e80830fab064ef3190811b022680a0aba4614d7e0e95e90a6268c6b: Positives: 0, Total: 56
	ed333742b1d328e83a2eb2610d94b1ac70b6f88a40b978d0683502b819d45285: Positives: 0, Total: 53
	ec904beca8b268a4a26ec09d32614e4064698b59dc2df848b22eac4f5a49f0c9: Positives: 0, Total: 55
	eb9ca996df33909ab25b98e033d820cf0b687d7d833d38e4948749163ed60c10: Positives: 0, Total: 53
	e953ac3b639202cfc647a0ab36599f45a678161be47789c7cf3c2132177e5f44: Positives: 0, Total: 55
	e6755e04f472f478684e6fec9226f7fc82fe0576b6e0ae7504ffcbb41832cb5c: Positives: 0, Total: 54
	e220b8b5afe2745bd3a92d1d961fe5bb7bc06b02a0046c7a9e3bde06b8e2ad02: Positives: 0, Total: 53
	e1f818767ba2c60a77d172da8bb31fd6e46a7291331568c00fe59877012b55cb: Positives: 0, Total: 54
	e17a0261a12397547696519d748e0756d95c2fe694fa8399179a3aaad4f075cb: Positives: 0, Total: 53
PassiveTotal Data (top 10, sorted by lastSeen). 	First Seen: 2011-02-12 13:38:44. Last Seen: 2021-11-05 09:52:46. Records: 55
	LastSeen: 2021-11-05 09:52:46. FirstSeen: 2021-05-25 11:31:39. Hostname: sentri360.ai.
	LastSeen: 2021-11-05 08:07:03. FirstSeen: 2021-06-11 04:15:47. Hostname: yunxuetang.ai.
	LastSeen: 2021-11-05 08:05:51. FirstSeen: 2020-01-08 20:33:20. Hostname: tant.al.
	LastSeen: 2021-11-05 07:57:48. FirstSeen: 2020-11-02 07:10:20. Hostname: salgado.com.ar.
	LastSeen: 2021-11-05 07:43:27. FirstSeen: 2016-03-08 21:26:03. Hostname: lewicki.com.ar.
	LastSeen: 2021-11-05 07:25:37. FirstSeen: 2020-12-06 20:19:25. Hostname: azmedia.com.ar.
	LastSeen: 2021-11-05 07:24:52. FirstSeen: 2021-06-30 18:20:49. Hostname: prueba.cammesa.com.ar.
	LastSeen: 2021-11-05 07:20:14. FirstSeen: 2016-02-09 17:22:21. Hostname: df.eaglemobile.al.
	LastSeen: 2021-11-05 07:12:50. FirstSeen: 2021-06-13 12:42:09. Hostname: test.trovo.ai.
	LastSeen: 2021-11-05 07:06:27. FirstSeen: 2018-05-20 00:00:50. Hostname: links.rakbank.ae.
	LastSeen: 2021-11-05 06:43:54. FirstSeen: 2021-02-17 01:16:25. Hostname: kgs.am.
Shodan Data. 	Tags: []
	Domains: ['one.one']
	Hostnames ['one.one.one.one']
	Org APNIC and Cloudflare DNS Resolver project
	Last update 2021-11-05T17:58:53.742055
	Ports [80, 443, 53]

IP Enrich can be imported from another Python as a module. An example of how to do it is shown below:

#!/usr/bin/env python3
import ip_enrich

ip = '1.1.1.1'

ipobj = ip_enrich.IP(ip, 10)
ipobj.getAll()
print(ipobj)