Other than some menu cleanup for 26.3, this version adds an even more exciting "dynamic binding" mechanism that is tied to the ESP32 Chain SMART Buttons. As mentioned, the SMART Buttons encompass all of computer science - as defining the button, and running the button are everything. It's one thing to hard code the definition of the button, but it's another thing totally to make that definition changeable and dynamic. (see https://SemanticMarker.org/images/SMARTButtons2026.jpg) This new version supports 2 new messages: "getSmartDefn, uuid.flownum", and "smartDefn" passing the button definition as "uuid.flownum(set,val,dev)". This is translated to the MQTT messaging which can be retrieved from Semantic Marker(R) SMART button definition (currently in the 'future' field). (The tuple of set, val, dev - are for defining MQTT parameters, or dynamic parameters). These are defined in the https://Github.com/konacurrents API manual. Two types of SMART Button definitions are supported, with one being dynamically changed through the DOCFOLLOW message. Kind 1: The first is a straight SMART definition using the uuid.flownum. The getSmartDefn is then sent out to anyone listening, and the Semantic Marker (R) app hears and returns results (if known). The reply is the smartDefn call. Clicking on the button (single press, double press, or long press) will invoke that MQTT message. Kind 2: The second is a DOCFOLLOW (or DC) definition. This awaits a DOCFOLLOW (or Semantic Marker) message from the Semantic Marker remote education features. The current DOCFOLLOW leader automatically sends a message when a new Semantic Marker(R) is found. Then when the ESP32 hears that type of message, it looks for any DOCFOLLOW SMART Button definitions awaiting definitions. These are then turned into SMART definitions, and Kind 1 is then run (calling the app and getting replies back). To run the DOCOLLOW, the knowledge shark subsection can be used.

For 2026 a year year, a large focus will be on accessing our suite of IoT devices especially for my #Ability users. To that end, I've added more Speech input and Spoken Output. The Matrix can be spoken out so my sight impaired users can decide which SMART Buttons to touch. And with Speech Input - they can now say "cell 5" to fire off that button (defined through a Semantic Marker(R)). When wrapped in 3d printing things come alive (even just Braille to find the right SMART Buttons to press - eg. scan) Also for 2026, a renewed focus on the Remote Education potential. The DOCFOLLOW model lets users take control of a remote users content through "optical" recognition of a Semantic Marker(R) and sending messages around to other users in say a classroom. Instead of the ZOOM paradigm of sharing all the bits of a users display, this version is light weight (bandwidth) where only the meta-data representing the Semantic Marker(R) are shared. The recipients can then have their displays updated to the page with the same optical visual marker - a Semantic Marker(R). Note this also works if playing back class notes while NOT synchronized in a ZOOM setting. Instead the notes are triggered by what I call the "Vision Linker". Also, with an Inheritance concepts, the Semantic Marker(R) might be your version for homework assignment 5, so your content takes over. Finally, the Semantic Marker(R) is being morphed into the "sync" end of our LinkAndSync.me concept. Typically a Semantic Marker(R) is a pointer, hypertext, hyperlink to other resources. I use it to point to a web link of cool stuff, a pointer to social media content, or a pointer to my on-line photo albums. Here the "link" will point to that resource, and through the indirection of that link can change to a different end-point. But how does that get updated? For 2026 I've add the "Sync" concept were an offline program would run on your server (where your sync artifacts exist) - and will automatically traverse your artifacts looking for markers (and especially sync versions - then end points). This will then update the indirect link database so your previously handed out optical visual marker - will always be up-to-date. This leverages the "Vision Linker" at a whole new level. It really extends the DOCFOLLOW model described above. As mentioned, 2026 is the year of IoT especially for my #Ability users - those that have troubles with existing computer interfaces, even the tactile button click of an iPhone button isn't easy. I have Kevin using Eye-Tracking to click on those matrix buttons. Others can use the scanner to find optical markers - even having them speak out to help the sight impaired users know if they are on the right set of Buttons (This is the TRON feature). The M5Stack suite of IoT devices are being extended with more physical "Buttons" that will map to the various SMART buttons in the virtual space. Let me know any special needs you think I can help with and we can morph the Semantic Marker(R) to your needs. Contact me for access to the IoT suite of devices possible. Look at SmartAnimalTraining.com which is designed for compatible with this app. Oh, almost forgot, a Semantic Marker(R) can be created that takes the user to a specific point in an audio resource - a podcast. Now you can point to hour/min/sec 2hr, 36min, 22 sec in a podcast (or any mp3 resource). Try doing that without button clicks etc - what we are trying to avoid for #Ability. I'm also trying to get IR and RF remotes to work so you can scan a QR code and the TV turns to the weather channel (no matter where you are, so the devices learn you location and those various TV removes). Stay tuned .. or if you can help let me know.

The best new feature, tailored for the sight impaired or anyone, its the default "speaking" when decoding a Tron Semantic Marker(R). see https://semanticmarker.org/sm/ Scan that .. and it will speaking the words (such as "Scott says control living room lights"). These are encode in the QR Optical Marker part - without having to go to the web for decoding. That contrasts with SMART which use the address to contact the Agent in the cloud to perform or retrieve more information. Note: your iPhone has to be non Silent mode to hear "Tron" talking. Or if you have a bluetooth speaker it will work. Currently there a strange bug where the Map collections are treated as a group collection - and it crashes the app. The location is outside my debugging (as it's in the Apple library somewhere). Very strange to overload a type without my code being able to know the difference. This error shows when displaying all your QR codes on the map - and then moving around to other pins. The cool feature of my Semantic Marker(R) is that you can scan a set of markers and once digitized, they are first class objects. You can get a map of all the markers you scanned - the map specified by the SMART Button designer. For example scanning my music marker with Paris, London, etc. A map shows up after scanning the markers with the Vision Linker turned on. https://semanticmarker.org/users/common/MusicMatrixSM.png Scan this matrix .. and then you will see a map (pulldown "show Matrix")

This version 42.1 will store the latest Vision Linker constraint matrix (the 4x4 blue box you see when using the camera). The constraint engine looks at pair-wise relationships of SMART Buttons as they are laid out by the user. If you scan a 3x3 matrix of buttons (the Semantic Marker images) - the Vision Linker(TM) should be able to recreate that matrix through the digitization process. see https://www.whiteriverranch.com/Groups/OnLineTechMeetings/MarkScottTalks/Contextual-Reality-Aug2025/i-wxqK4gM/A Now that relationship will be saved for next time, which can be useful if the app crashes - since different phones deal with the memory allocations different. Now you can re-start the app and bring up the camera, the it keeps the previous scanned matrix of objects. This same matrix can be shared with others (as text) or in your cut buffer. You can bring in a shared matrix through the cut-buffer of the phone (with a menu in the camera). Another interesting aspect is in the MQTT scrolling window. You can now see images that represent status messages. The same Augmented Reality will be shown in the scrolling window. Also, as done in 2022 for the M5 ESP-32 device, the status message contains information such as the battery level, lights on/off, buzzer on, etc. This can be shown graphically after sending the #STATUS message. In addition, the Object Oriented aspects of the ESP32 architecture is exciting. Now the same build can support numerous versions of the plug-compatible parts - and chosen at runtime (so they are linked in). This means one M5Atom device might include the GPS unit and another the M5Camera. Through messages to the devices, they morph to that functionality. In addition, some devices need PIN numbers described. That is all supported. See an OO diagram at: https://knowledgeshark.me/docs/ESP_IOT/html/dir_0c6ecdc73d638af91cbc44cea9a62df0.html Additionally the M5Camera can provide a local RTSP stream, and also upload an image from the camera - which using the #url message, will show up in the scrolling MQTT window. Note: the Bluetooth namespace can get corrupted it seems. So I might have a device called M5AtomSocket, but it thinks it's just M5Atom or M5AtomGPS. It's very strange. It seems names linger in the Bluetooth Advertisement namespace. As always, please contact me for any 1-1 time to learn how to create your own Semantic Marker. Using the PDF printing as the tool follows your SMART Button hierarchy and inheritance tree is exciting. The format for the printout is simple for now, but could become very rich with information.

8.15.25 CONE Tree with Space & Time # CONE Tree PDF: This version is exciting as we are getting back to auto-generation of PDF artifacts. But this is with a twist as the Semantic Marker(R) contains a hierarchy definition where a SMART Button might have many sub-buttons. These are traversed manually in various ways, but it shines when the tree/graph is generated. I call this the CONE Tree after the 1988 CONE Tree, a 3d image showing the full tree, and shadows for parts you cannot see. It was impressive: https://SemanticMarker.org/docs/conetrees.html Asside from the last release of the Avery labels, the new verison traverses this graph of nodes, and generates a PDF book. The template contents are still being designed, with information, like 'description', extracted for the users SMART Button instances. This PDF can be shared in the usual iPhone ways (email, text, airdrop, print). # Remote Eduction: I've included a couple versions of my Knowledge Shark Computer Science architecture book, still in draft format. But to showcase the Knowledge Shark remote education it will be useful. This is the DOCFOLLOW feature that uses messages to share the Semantic Marker(R) which are used to direct each others information visuals (auto turning pages, etc). Example PDF: https://SemanticMarker.org/docs/SemanticMarkerPDF_NDA.pdf The ultimite goal is to leverage this framework for "contextual aware" multi-media, both in the virtual on-line environment, but also in the physical (paper) world. How does the "Book of TomorrowLand" look? I'm working on a podcast segment to showcase exactly this - a "run-able" book. Think of your new dishwasher appliance that's all SMART - so a message can change the settings on water temperature. This manual will be in a paper format, and other thans "words" that state the buttons to press - there won't be any automation. Sure the "app" once you download, and configure with account and WIFI credentials would work. But that paper manual is just that, paper. There is no "run-able" nature to the paper. # Semantic Marker(R) Game Changer: That paper manual can now run from your smart app. After your have logged into your own app the scanning of the paper can now "run" - thus "runable". It might change the dishwasher water temerature, turn on the xmas lights, or feed the dog. But importantly, and "contextual aware" - this is in the local environment. For example, say an building has 100 idential appartments with the same appliances. The printed user manual should be identical, but obviously the water temperature page should only change appartment 5's dishwasher. But if you went to apartment 10, it wouldn't work - unless you had been granted permission. So from the paper manual (or digital) - with the right credentials - the paper user manual comes alive - and is "runable". # M5Atom GPS and STATUS (IoT): The IoT Framework at https://github.com/konacurrents continues to advance, especially as related to this app. There is a mode where graphics are used as part of the Augmented Reality graphics shown by the app. Here status information is shown in graphic format by decoding from the messages sent by the M5Atom ESP-32 devices (through our message framework). The IoT devices are key to the "runable" book - as it requires edge nodes to perform the operations at the users request. So the water temerature remote control is an IoT device embedded in the dishwasher, and connected to the messaging infrastructure. As each dishwasher is idential, they have to be instantiated in the users environment (eg. Appartment #5 vs #10). And they need credentials for the users, etc. Thus this Semantic Marker(R) app provides the foundation for "context" aware smart books. Much like my favorite article from Robin Milner, a Turing Award winner while also talking about his friend, Turing himself. https://SemanticMarker.org/docs/milnerTurning.pdf Let me know if you need any 1-1 time with the app.

This version finalizes the Avery 94227 3x6 labels, so you can print out a set with the full Semantic Marker(R) in the first 9 and just the QRAvatar Image in the 2nd 9. Paste them back to back - or stick them into a blank form or on products you hand in as homework (denoting your version of the professors homework). Various Menus have been organized based on usage, and more help menus where added. New for Summer 2025 is that SMART Podcast Clips (Qwinton's idea). This is an extention to the SMART Music player, but instead of just a playlist, now the user can specify "clips" which are start and end times within a full 3 hour podcast. The player automatically advanced to the next clip. The syntax extends the mp3 URL with a start and optionally end time (eg. path_to_music.mp3-1:22.3-2:22.2 -> meaning start at 1 hr 22 min and 3 sec and end at 2:22.2). A sample is at: https://semanticmarker.org/bot/smart?uuid=QHmwUurxC3&flow=1679615480376 where I talk about 3d SMART Passwords. This is still in-work but usable as it stands. In the SPACE and TIME - users can now specify when they have seen a Semantic Marker(R) and a GPS location (even copied from a users own photo's GPS info). Using the SMART web page, the owner of that Semantic Marker(R) can then absorbe that location or just note it's sighting. Also in SPACE and TIME - when combined with moving objects, especially the new GPS extension to the M5Atom ESP32 SMART Button, the location will show on the map and move as the button moves. Think of a SMART Dog Collar - assuming the owner can provide a WIFI hotspot. The GPS is from Satellites. Finally in SPACE and TIME - any Semantic Marker(R) that uses the GPS and time features, when a user exports as an Photo Image - the metadata (EXIF) will denote that GPS and Time information - and also create a caption with the full Semantic Marker(R) address (so the photo, or full Semantic Marker(R) with photo, now can be run as if a QR code scanner determined that address). Try out the DOCFOLLOW features for remote eductation, where you or another leader, controls followers displays changing to the pages with the corresponding Semantic Marker(R). This Vision Linker(TM) feature has untapped potential, especially for the #Ability market. The perfect companion to the Semantic Marker(R) are the apps build with the ESP32 framework. We have a number of devices made with 3d printer, and integrated with the M5Atom and other Sensors (such as the GPS receiver or the QRCode scanner). This is what I've called iDogWatch.com since 2014, now with a production runtime framework. WIth an account, you can access all the messages and create Semantic Marker(R) SMART Buttons. See https://SemanticMarker.org/smart See the Open Source version of the ESP32 based software: https://github.com/konacurrents/SemanticMarkerESP-32 These follow the API defined: https://github.com/konacurrents/SemanticMarkerAPI As always, let me know if you want a personal demo. I also have cool Kona Currents coffee cups for sale.

An exciting new Semantic Marker(R) now really supports Space and Time: your own SMART Button can be tagged with geospatial (GPS) values so they show up on map products at the desired location. You can also enter time values in the past, to reflect the SMART Memory, and also into the future. This will be a future triggering mechanism. But what's super powerful is that a Semantic Marker(R) can be sent through the air, maybe in a paper airplane across the late Kingdome (with no electronic tracking of that message). Then re-digitized on the receivers end - and they will get the same location and date: Space and Time. So a high school reunion event, or photos from 50 years ago, are tagged with the correct location and time. Then the app can show a map, but also just photos saved to a users smart phone photo library, will have those values in the GPS and EXIF photo values. Thus outside of my app, you can get mapping and time features - since exported SMART Buttons will include those values. Truly portable (outside my app). Triggers are also incorporated better. This now supports Location Triggers: the ability for the GPS mentioned to be triggered if a user is within a distance of that Semantic Marker(R). This is without bluetooth tags, and instead using the GPS of the phone. Other triggers are based on MQTT set JSON values. Thus a trigger might be for the feedCount to be greater than 100. Currently a DOCFOLLOW message is sent. Example Apple map of Grateful Dead albums mapped to their concert location, and a timestamp of event. Again this is outside my tool once my app exports the markers as optical or visual markers. https://photos.smugmug.com/Groups/KnowledgeShark/SemanticMarkerUserManual/i-nJ7q6SN/0/LxzwwNJr5RtSCHpQJBWdMWG6jB6Bn3474JFvBQTvn/X2/IMG_4707-X2.png Some tools and web sites strip the location and time, as an example with the following (which will take the user to a SMART Music site for a 1973 Paris Grateful Dead concert. https://semanticmarker.org/music/May2_72.PNG TRON - the SMART Memory Stamp is also powerful. One can now wrap TRON in a Semantic Marker(R) so they are stored (and can change). I really like the SMART Music aspect of the Semantic Marker(R) app. Here you can point to music artifacts on your phone, and create a SMART Button with the image of that music and links to the requested music. But what's even more interesting, is that the archive.org open source streaming site, can now be referenced and my code will extract the playlist including the song names and a link to the mp3 of said song. This is placed in a SMART Button in the artifact field with @playlist and @artist fields. As mentioned above, a MAP can show where a Semantic Marker(R) selection exists in Space and Time. But also photo images stored on your phone can be used in a similar manner (mapping where they are). Think of a relator with locations of all their houses. Now the Semantic Marker(R) will have that GPS information, as well as time, and any artifacts with the SMART Button itself (the web page for the artifact.) Note the Semantic Marker(R) offers a 'markup' field that will result in a web page being rendered for the user - without their own web page storage. The potential for getting use counts from random potential clients (say at the Puyallup Fair) will also be introduced soon. I've collected some videos that loosely form the user manual: https://www.smugmug.com/app/organize/Groups/KnowledgeShark/SemanticMarkerUserManual Let me know if you want more information on a new way to process our information and our memories. We coined "Grandmas Attic" as one application. This is where the physical and virtual world overlap.

We are exploring uses of eye-tracking with our app. This is working out nicely especially with the matrix and Vision Linker. Brand new is our TRON - SMART Memory Stamp. This version of our Semantic Marker(R) can be created that is fully self defined so it can be decode without the internet, or you can also still use the internet. See https://SemanticMarker.org/sm - but the power with this app is that scanning a TRON Memory Stamp will show an augmented reality graphic overlay with a short amount of information (see the promotional images). This can be useful when many markers are grouped, but you want to know what the group is about: add a TRON memory stamp, and immediate feedback will show a couple lines of text. As these are self-defined, they can be cloned, modified, etc - without any persistent storage; the optical visual marker is the storage. We are starting to showcase the grouping of Semantic Marker(R) images into a matrix. These can be created with our tools or can be dynamically created by scanning sets of optical markers. The digitization of multiple Semantic Markers in the physical environment - printed pieces of paper - use a custom constraint engine of pair wise locations (a Vision Linker). The result is a matrix digitized into the virtual world - and from then on you can be virtual. But that matrix of markers can be easily converted back to the physical environment for sharing, printing, email, etc. This symbiotic relationship is unique and why Hypertext (described below) is making a comeback when in the physical or virtual world. The graphic images, our optical visual marker, are "infinitely portable" By this I mean that they are still valid in almost any format and any medium. As I've been publishing various aspects of the Semantic Marker(R) architecture, these include demonstration images about the concept. The trick is that all of these images can run, be invoked, by anyone in the audience in each medium (printed physical, virtual). This powerful symbiotic relationship is the power and the curse since they all "run". Thus some of the promotional material have a black bar to not let you feed my friends dog for now:-) This symbolic relationship is powerful, as the optical markers can transform from the physical (paper) to the virtual (digital photos) - and back to physical (paper) as often as needed. Remote Education is being explored more. In particular, I've included a few Knowledge Shark "waves" such as a version of my draft book and other slides. When teamed with the messaging capability, remote users can drive your screen (and visa versa). Watching a disabled user drive the collaborative education viewing is fun to watch. I call this DOCFOLLOW and you can be a follower or leader. When a leader, changing pages in a "wave" (basically a powerpoint multimedia collection) - will synchronize others in the group to change their display to the same page. But most important invention is that this "page changing" is via recognizing a Semantic Marker(R) on the current page - and then the followers change to their version of slides with the same Semantic Marker(R). This is unique. It's basically digitizing the optical marker and sending that across the network to the group. Then they un-digitize and find that same marker (which was digitized locally). This even works with my "inheritance" model. This means you might be working on you homework for problem 5, and have an inherited Semantic Marker. This page changing would then use either the inherited or original marker. We are also working on a Braille version where a 3d printed card holder might have words that help the sight imparied find a matrix containing paper printed Semantic Markers. One set might be to turn on the xmas tree lights, while another to feed the dog saying its name. The user would scan the set desired (after choosing which set via Braille). This is versus exhaustive scanning everything. So it's a mix between physical and virtual processing.

Using the term "Custom Intelligent Products" or CIPs, this release is tailored to improve the user interface and make it easier for those using eye-trackers as their main interaction. This means streamlining some of the buttons. Making buttons bigger. The Matrix is a key tool where up to 9 SMART buttons (those that do things) can be shown at the same time and user configured. When connected via Bluetooth to an IoT device, messages can be sent very fast. Otherwise the WIFI and our iDogWatch, SemanticMarker cloud system relays those messages to the respective IoT devices. Scanning a Semantic Marker (QR Code) Still one of the most valuable uses of this Semantic Marker tool is configuring the various IoT tools we have been developing. This really shines when modifying credentials, like the WIFI passwords and other user information. This is done over bluetooth adding the intelligence to our CIPs. Also, if you have music on your iPhone, you can create a Semantic Marker that is a special playlist of a subset of songs. Then you "store" that list in your photo library as a QR Code with the album cover. Then invoke it later to add that music to your music playlist. I've also been using the 3d matrix password (a collection of Semantic Markers) when needing a new password on some sites. The shapes of the collection (like L, T, etc) are part of the password. Try to break that password. Note that our trademarked Semantic Marker (R) is now registered in USA, UK, Japan, Canada, and the EU. Exciting. 33.3 release  March 15, 2025 First quick important update, where the MQTT credentials were not being sent to the device. Instead a popup should have showed asking for confirmation. That window never showed up and nothing was sent. That's been fixed. Next, in working with Kevin and his use of Eye-Tracking, some things are challenging. This includes: a pulldown menu where you have to scroll past what is shown. Kevin has to change eye-tracker mode to a focus, or a swipe, or other. Anyway, some common commands (like reboot of IoT device) have been moved up to the top. Also, clicking on buttons - when there are buttons next to each other, is sometimes challenge. To help this, a menu pulldown has been added for sending MQTT and WIFI credentials (as well as the button). Next; I've included an early version of my Knowledge Shark book - but in Knowledge Shark 'Wave' format. So the 95 pages are broken into 3 streams and shown 2 pages to a slide. The included Semantic Markers have been pre-scanned so they will just be visible and work with the remote education DOCFOLLOW capability. Uses should be scanning any QR code they see out there, and the click on the "MAP" feature. Then show the QR codes from a pull down menu. I have various demos and some help information in video form. Contact me if you want to see any of them.

2025 is starting out nicely for technology and the new User Interface (UX) approaches made available, especially through eye-tracking. The latest Semantic Marker(R) App cleans up a lot of the pull down menus. Of particular interest are the various shapes you can arrange groups of Semantic Marker(R) SMART Buttons. This leads to what I call a SMART 3d Matrix Password. The user selects a couple or more images - remembers the shape (like L or T) and a textual password is generated. This is then used in a password (even adding special characters.) These are "strong" passwords that can be remembers as "visual" images (eg. Rabbit upper left, Dog middle, Horse bottom left.) Exporting sets of markers is also enhanced so you can print them double sided, where the "optical marker" is on one side, and the "visual marker" is on the other (as us Humans can only make sense of the visual marker.) These can be cut out and made into business cards (ala Taro cards.) These all run. The advantage of the Matrix of SMART Buttons, is that common commands and Internet of Things (IoT) can be run through these buttons. But since they are bigger and easier to "touch" than the menus or other traditional user interfaces, they are more accessible to disadvantaged users. This is especially valid for the eye-tracker users - clicking these buttons through their eye-tracker. It's amazing to see this in action. One client, Kevin, has our dog feeder bolted to his wheel chair, and his dog tethered to the chair. He then eye-clicks on a button to say out loud: "Abby heal". Then if the dog does the command, another eye-click invokes the messaging from this app (or a web version) and food rewards the dog. Really "eye opening". A couple other features: A Semantic Marker for various WIFI can be stored within the app, and also shared via Bluetooth to your IoT devices. And when you scan a couple of them the Augmented Reality will show the difference, helping the user decide which one to use.

New menu and cascading menus have been incorporated. This makes accessing functionality a little easier and more consistent across the app. Also the button press on the SMART Matrix and SMART Camera provide better feedback, including optional haptics. Thus clicking a button feels like clicking a physical button. Many of these improvements are trying to support Eye Tracking and other #abilities capabilities. With the iOS 18.2 release, a new photo "Utilities" menu shows your "Favorites" but now through analyzing photos, new categories are supported. In particular "QR codes" are now exposed (I have 532 so far). This also includes includes "Receipts", "Handwriting", "Illustrations". In addition, these can be shown on the "map" of where the photos (with a QR code) was taken. Note, for the photo analysis to work, the QR codes must be "in focus", otherwise these won't be found. My app has an option where a photo is automatically taken when the QR code is "in focus". The Semantic Marker(R) App has incorporated these features and more for at least 4 years. This includes maps, a subset of photos showing just QR codes, etc. But the best the photo app can do it invoke the "web" version of the QR code. The Semantic Marker(R) App has a much nicer integration with your SMART Agents behind the "optical visual marker" - the QR Code. So I can browse photos, or browse my SMART Buttons, and clicking on one might turn on/off the christmas tree lights (and either over MQTT or Bluetooth BLE.) What would be powerful is for Apple to incorporate these Semantic Marker(R) executable functions, including the Augmented Reality, Agent Processing, and the like - and make them an add-on to the photo app. Let me know if you want to learn more and also if you can help me find a way to exploit the Apple iOS 18.2 admission of the power of these QR Codes.

A Private Semantic Marker(R) is now supported. Scanning these will require a password to access the web page, or invoke messages. The QRAvatar is still available but nothing else. Once opened, the Semantic Marker could have other markers that contain say the key to the front door of your rental. The owner can change the password anytime, as well as maybe the key. The password can be complex as described next. Use the Vision Linker built in the SMART Camera - and digitize pairs of Semantic Markers. This will re-create a matrix from the physical world into the virtual world. As images (our visual markers) are powerful for humans, a side effect of the SMART Matrix is the creating of a password, what I call a 3d Password. This means the password contains the Semantic Marker address, but also the position in the matrix (eg. 1.1 or 3.3). Adding a personal string can help too. This password will be hard to break and the user has the potential to remember the password just by remembering images (your QRAvatar for the SMART Buttons) - and remembering the configuration. This powerful Vision Linker Constraint Engine can create a powerful password. It uses a syntax that can be sent on a URL or stored in a SMART button (using the new privatePassword.) This is unique in that you can remember say 3 images (dog, cat, horse) - which are in an L shape. You know the dog is a SMART Button as are the others. The password will be the Semantic Marker Address and the row/col in the L shape (or whatever ever shape you like.) The Vision Linker also supports the DOCFOLLOW paradigm better. The Knowledge Shark waves (or remote education slides) - will synchronize remote users via scanning (or finding) a Semantic Marker on specific pages. The Bluetooth BLE messages are more powerful in that most of the JSON messages used in MQTT, can also be sent via BLE. You should see the ESP32 light switch toggle on-off via bluetooth, and much faster (many times a second). This is from scanning a Semantic Marker. Group processing is also powerful, and will be the Air-BnB or VRBO solution for secure use of your SMART house for a short time span. The user manual you provide will have a Semantic Marker for each of the SMART Button operations (lights, dog feeder, etc). But the manual only works if a valid user is authenticated. Think of what you would do once you have multiple Semantic Marker images? How do you keep track of them? How do you create a hierarchy of markers? Semantic Marker(R) provides that organization. Best of all - you can create and capture all this power just by creating a photo containing what we trademarked as an "optical" "visual marker". The optics are read by the computer., while the visual is read by the human (us). The power of the new release is with remote triggers. All SMART Agents have the capability to send numerous IoT messages. Now, with triggers, agents are truly treated as background processing agents. To enable these, a trigger syntax is now used. This will be a simple conditional (like name = scott) for starters. When the trigger is fired, the IoT messages are sent (after configuring for the user and device.) Also, the Hotspot is supported where QR Syntax hotspot Semantic Marker(R) can be used (and the app lets you create your own with the valid syntax). This is also used as credentials to send down to the IoT devices via BLE (Bluetooth). # Digitizing optical markers This version continues the digitization capabilities of the Semantic Marker(R). This means you can place a number of Semantic Marker(R) images on a table - organized roughly like a matrix (or sparse, maybe an L or T shape). The digitizer will create a SMART Matrix from the images. Also, the MQTT window will show messages as they arrive, and when they are SMART DOCFOLLOW messages - they will be shown to the user. This way you get feedback on the group collaboration. Again, let me know if you want more information. I'm available for 1-1 collaboration.

This exciting version dynamically updates the users/devices you have running and presents it in various Semantic Marker (R) SMART buttons. These basically absorb the environment through listening to MQTT messages and updating values represented by the Semantic Marker Augmented Reality images. The BLE bluetooth messaging is updated to support the latest ESP32 devices (see https://github.com/konacurrents.SemanticMarkerAPI). The SMART Matrix is updated to be interactive and eye-tracker aware (with 3rd party eye-trackers.) An exciting new feature is the SMART Vision Linker which will allow creating of a SMART Matrix just by scanning pairs or up to 4 Semantic Marker's just placed on your table top. Positional relationships between the markers creates the matrix using the Vision Linker. This augments the relationship between a Semantic Marker(R) in the physical and virtual spaces. The new matrix is shown in the camera view as the constraints are satisfied by our Vision Linker. This matrix can be shared over the DOCFOLLOW messaging (MQTT) so followers could see results as they happen. Just think of the games this could foster! Another powerful feature for our less iPhone aware users is a new kind of "motion sensor" where our Semantic Marker(R) aware camera is put in "selfie" mode, and placed on the arm of your chair. Then an appropriate Semantic Marker(R) is waved over the camera and when "in-focus" the Augmented Reality overlay will show up - and the action performed - such as feeding your dog via an IoT message (or over Bluetooth BLE). This could be more accurate than just a motion sensor (which our ESP32 M5 supports) - as this will only run when decoded correctly. (Think of scenarios where maybe a user has a magnet device on their hand, and after picking up a magnetized Semantic Marker(R) - then wave it over our app. Maybe another marker is picked up next to repeat commands. And with our spoken messages and IoT messages, the potential is unlimited.) Next, each scanned Semantic Marker (R) will record its GPS location and let you create a map of those scanned items. These can be exported in KML format for Google Maps/Earth display, or internally with map app. Think of the Puyallup State Fair. Finally, the secure Group membership is integrated throughout. This supports the VRBO/AirBnB shared smart house (or guest kennel) where you grant a temporary guest access to some of your smart house devices (but not your safe.) Then users use their own account and operate the appropriate IoT devices (TV, Lights), and when done, the owner removes their membership. Even more exciting is the SMART user manual you provide to guests contains traditional how-to words, but now you can add optical visual markers - the Semantic Marker(R) throughout. These will run the smart house when scanned - but ONLY if you are a member of the group specified. So even if you took the manual home (or took pics of it), it won't work without those membership credentials. Take that Spy-vs-Spy. (Also your devices can opt-in to a subset of groups using the "includeGroups" message - so your safe could be on the "secureHome" group.) Let me know if you want a demo or need help getting things working. Thanks. ps. I almost forgot - DOCFOLLOW - the key to Remote Education/Training is also more integrated (with buttons to turn on Leader or Follower in the camera.) The exciting and novel aspect is that a DOCFOLLOW message is sent from the leader as a Semantic Marker(R) address. When received over MQTT, if in follow mode, the users screen will advance to the page where that Semantic Marker(R) is located (automatically.) Much like a lawyer document where they say to go to page containing footnote 5, not to page 5. Our version uses image recognition to find where the associated Semantic Marker(R) resides and travels to that page.

This version now supports marking counted positions of objects using GPS and Semantic Markers(R). I counted the 200 hay bales and have a KML map of those physical objects. The map feature of found markers is powerful - as you map your Puyallup Fair experience. Next: one can create a matrix of SMART buttons by just scanning the 3x3 (or so) markers placed on the table. Scan pair wise to find the relative positions. These SMART Buttons are interactive (touching does things, but also with MQTT the button can listen to message topics - and then re-draw on the Augmented Reality overlay while scanning.) I can now see the 1.5 million WIFI feeds count shown on my iDog'Watch' companion. The app shows those SMART buttons while using the Camera as well (not Augmented Reality, just virtual Graphics). DOCFOLLOW - my remote education tool - is expanded to the Knowledge Shark Wave. The slides in the wave are "optically" analyzed looking for markers. Any user in the same "group" topic can work together - someone sending "leader" messages, while others are not leaders - which is changeable (Like sharing screen). The MQTT messages are send around - and the remote users have their slides changing automatically to the same slide. But: to the slide that the Semantic Marker(R) is found on the leaders slide. (Inheritance is another feature I'll tell you about later..) The syntax for the WIFI in a Semantic Marker is already supported, but it's to set the WIFI hotspot of the scanner (running the app). There needs to be a version where the WIFI (SSID and PASSWORD) are added to the BLEConfig WIFI -- as this is to be sent down to the ESP32 device. The SMART Music app is taking on form and a life of its own. One can create a SMART Button that has a playlist, pointing to "mp3" files internet available (eg. public IP address.) The songs, or speeches, or videos - can be queued up and started by scanning the SMART Button. These can be placed in a SMART Matrix as well. You can create a a music playlist that is encoded in a SMART Button (and a SMART Matrix). Clicking invokes a web page built to play music. try: https://semanticmarker.org/bot/smart?uuid=QHmwUurxC3&flow=1718381656320 New app Help videos are available such as: https://semanticmarker.org/bot/smart?uuid=zWZUNFoclmYd&flow=1716483990995 The documentation on the messages supported is available at: https://GitHub.com/konacurrents/SemanticMarkerAPI

This version is exciting as it brings the iDogWatch features to the forefront. Namely, SMART Buttons show up on the SMART Camera - so you can click a button that says "Abby, Sit" - and while watching, or using video recording - if the dog performs that command, you can then hit the "Feed Dog" button - still on the same screen. The buttons show up on the left of the camera - and are defined using the latest SMART Matrix which is user defined. Also, the Semantic Marker for remote eduction is getting closer. This means while collaborating with others, the visual artifacts (like slides, pdf pages, etc) can be remotely changed by the leader turning to a page/image with a Semantic Marker visible. This tells others to then switch to the same page (or artifact) that has that same marker. I call this DOCFOLLOW. Note, a Google Maps KML file is generated from your scanned Semantic Markers that have the same location information (if allowed) that the camera uses. This map is then all the locations you visited at the Puyallup Fair - for example. The starting screen requires a new user to "request an account". Entering a user password and simple password will let us manually create that account and email you when completed. Once done, you can send messages to your IoT devices, create your own Semantic Marker(R) images - and best of all create SMART Buttons which do things (even play music). We have one client in a wheelchair who can only type and use the computer through eye tracking (not our product). This renewed Agency lets him reward his service dog through the Internet of Things messaging provided via our Semantic Marker(R) app. * SMART: Semantic Marker(R) Augmented Reality of Things (Internet of things)

This version lets users request their own agent account in the cloud. This will unlock more features of the Semantic Marker(R) app. In particular, you can create your own SMART Buttons, and change the dynamic functionality, the Augmented Reality overlay, and other aspects. This helps the human use their own Visual Recognition (even eye tracking) to make sense of button images. The Semantic Marker (R) toolset uses the associated Optical Code Marker to make digital sense of your markers. (For example, send a message to your IoT device to turn on the lights or feed the dog.)