[SOLVED]LAMW - accessibility setting- Floating - Service -jUSSDService lazarus / 43-Android ---------------------------------------------------------------------- Guser979 · 2021-09-20 23:35 Hi. I have modified the JUSSD component to work as an accessibility service which lists package name and all currently active screen content. I used this service together with jWindowManager . After a lot of testing and modifications now everything works perfectly as I imagined. Only one detail is missing: I want the application to open the accessibility setting so that I can activate the service on the first run. Thanks in advance. PS: My intention with this one is to automate tasks. There are many, I know. But nothing like having a custom one. And with LAWM this is possible. EDIT : Done with : In modified Jussdservice .... import android.provider.Settings; Intent intent = new Intent(Settings.ACTION_ACCESSIBILITY_SETTINGS); controls.activity. startActivity(intent); Code Code: Java [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i package com.example.appwindowmanagerdemo1;import android.provider.Settings;import android.accessibilityservice.AccessibilityService;import android.accessibilityservice.AccessibilityServiceInfo;import android.content.Context;import android.content.Intent;import android.os.Build;import android.text.TextUtils;import android.util.Log;import android.view.accessibility.AccessibilityEvent;import android.view.accessibility.AccessibilityNodeInfo;import java.util.Collections;import java.util.List; import android.accessibilityservice.GestureDescription;import android.graphics.Path;import android.graphics.PixelFormat;import android.media.AudioManager;import android.view.Gravity;import android.view.LayoutInflater;import android.view.View;import android.view.WindowManager; import android.widget.Button;import android.widget.FrameLayout; import java.util.ArrayDeque;import java.util.Deque;/*Draft java code by "Lazarus Android Module Wizard" [7/14/2020 16:47:51]*//*https://github.com/jmpessoa/lazandroidmodulewi zard*//*jControl LAMW template*/ // ref. https://stackoverflow.com/questions/22057625/prevent-ussd-dialog-and- read-ussd-response// ref. http://habrahabr.ru/post/234425/ public class jUSSDService extends AccessibilityService { private Path path; private GestureDescription.Builder builder; private GestureDescription.StrokeDescription strokeDescription; private GestureResultCallback gestureResultCallback; private GestureDescription gestureDescription; private long pascalObj = 0; //Pascal Object private Controls controls = null; //Java/Pascal [events] Interface ... private Context context = null; /*.*/public static String TAG = jUSSDService.class.getSimpleName(); //GUIDELINE: please, preferentially, init all yours params names with "_", ex: int _flag, String _hello ... /*.*/public jUSSDService() { //super(); } public jUSSDService(Controls _ctrls, long _Self) { //Add more others news "_xxx" params if needed! //super(_ctrls.activity); context = _ctrls.activity; pascalObj = _Self; controls = _ctrls; } private String processUSSDText(List eventText) { for (CharSequence s : eventText) { String text = String.valueOf(s); // Return text if text is the expected ussd response if( true ) { return text; } } return null; } private String toString(AccessibilityEvent event) { String str = ""; str += "eventtype: " + AccessibilityEvent.eventTypeToString(event.getEventType()); str += "\nsource: " + event.getSource(); str += "\npackagename: " + event.getPackageName(); str += "\nclassname: " + event.getClassName(); return str; } //public static voidpublic void logNodeHeirarchy(AccessibilityNodeInfo nodeInfo, int depth) { //if nodeInfo.getText() = "Home" {// //currentNode.performAction(AccessibilityNodeInfo.ACTION_CLICK);// } if (nodeInfo == null) return; String logString = ""; for (int i = 0; i < depth; ++i) { logString += " "; } logString += "Text: " + nodeInfo.getText() + " " + " Content- Description: " + nodeInfo.getContentDescription(); Log.v("Meu", logString); Intent intent = new Intent("org.lamw.action.USSDService"); intent.putExtra("message",logString ); //text sendBroadcast(intent); for (int i = 0; i < nodeInfo.getChildCount(); ++i) { logNodeHeirarchy(nodeInfo.getChild(i), depth + 1); }} @Override /*.*/public void onAccessibilityEvent(AccessibilityEvent event) { //Log.d(TAG, "onAccessibilityEvent"); AccessibilityNodeInfo source = event.getSource(); if (source == null) { return;} String text= ""; if (event.getEventType()==AccessibilityEvent.TYPE_WINDOW_CONTENT_CHANGED) { if (source.getPackageName()!=null) { if (!source.getPackageName().equals("com.sec.android.app.launcher")) { if (!source.getPackageName().equals("com.example.appwindowmanagerdemo1")) { Intent intent = new Intent("org.lamw.action.USSDService");intent. putExtra("message","Package:"+source.getPackageName() ); //text sendBroadcast(intent); AccessibilityNodeInfo currentNode=getRootInActiveWindow(); if (currentNode!=null) { logNodeHeirarchy(currentNode, 0); } } } } } return; } @Override /*.*/public void onInterrupt() { // } @Override protected void onServiceConnected() { super.onServiceConnected(); // pag1(); // com.uieaser.atextwidget class android.widget.TextView id name R.id.textViewMain parent android.widget.RelativeLayout(R.id.RelativeLayout1) //Log.d(TAG, "onServiceConnected"); AccessibilityServiceInfo info = new AccessibilityServiceInfo(); info.flags = AccessibilityServiceInfo.DEFAULT; info.packageNames = null; info.eventTypes = AccessibilityEvent.TYPE_WINDOW_STATE_CHANGED | AccessibilityEvent.TYPE_WINDOW_CONTENT_CHANGED; info.feedbackType = AccessibilityServiceInfo.FEEDBACK_GENERIC; setServiceInfo(info); } public void jFree() { //free local objects... } //write others [public] methods code here...... //GUIDELINE: please, preferentially, init all yours params names with "_", ex: int _flag, String _hello ... //Method to start the service public void Start() { //Explicit intents explicitly define the component which should be called by the Android system, //by using the Java class as identifier //Create an intent for a specific component. Intent intent = new Intent(Settings.ACTION_ACCESSIBILITY_SETTINGS); controls.activity. startActivity(intent); } public void pag1() { if (path == null) { path = new Path(); path.moveTo(513, 1963); path.lineTo(513, 1963); } if (builder == null) builder = new GestureDescription.Builder(); if (strokeDescription == null) { strokeDescription = new GestureDescription.StrokeDescription(path, 0, 1); builder.addStroke(strokeDescription); } if (gestureResultCallback == null) gestureResultCallback = new GestureResultCallback() { @Override public void onCompleted(GestureDescription gestureDescription) { pag1(); } @Override public void onCancelled(GestureDescription gestureDescription) { super.onCancelled(gestureDescription); } }; if (gestureDescription == null) gestureDescription = builder.build(); dispatchGesture(gestureDescription, gestureResultCallback, null); } public void Stop() { controls.activity.stopService(new Intent(controls.activity,jUSSDService.class)); } } Add in JwindowManager (java) Code: Java [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i @Override public boolean onTouchEvent(MotionEvent event) { switch (event.getAction()) { case MotionEvent.ACTION_DOWN: SetViewFocusable(true);// add //remember the initial position. if (mParams != null) { initialX = mParams.x; initialY = mParams.y; //get the touch location initialTouchX = event.getRawX(); initialTouchY = event.getRawY(); } break; case MotionEvent.ACTION_MOVE: SetViewFocusable(true); //Calculate the X and Y coordinates of the view. if (mParams != null) { mParams.x = initialX + (int) (event.getRawX() - initialTouchX); mParams.y = initialY + (int) (event.getRawY() - initialTouchY); //Update the layout with new X & Y coordinate } case MotionEvent.ACTION_OUTSIDE: // add SetViewFocusable(false);// add XML Service in Res\XML Code: XML [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i

Sources: Many. Grateful to all who share knowledge. ---------------------------------------------------------------------- Guser979 · 2021-09-21 23:54 Well.... I "cloned' JUSSDService and now I have the new JUSSDService2 component with the modifications I need. Question : If I want to share just this component with someone, is there a way? I think it's not difficult to do something in that direction. I just need to reproduce what I did. Clone , modify, add to Register_extras and compile. I didn't use New JComponent from LAMW. I found it complicated ---------------------------------------------------------------------- jmpessoa · 2021-09-22 03:13 Quote I "cloned' JUSSDService and now I have the new JUSSDService2 Nice! But, the way to go is add your improvement to original component.... then I can update the LAMW jUSSDService. What about? ---------------------------------------------------------------------- Guser979 · 2021-09-22 08:55 Yes Master. I will try. I'll end up learning Java by the way kkkk... It's not that difficult. But I just think it's "boring" language. Hahaha Edit : My solution is not that of a programmer, but it works... hehe Here is the JUSSDService Code: Java [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i package org.lamw.appussdservicedemo1.App;import android.provider.Settings;import android.accessibilityservice.AccessibilityService;import android.accessibilityservice.AccessibilityServiceInfo;import android.content.Context;import android.content.Intent;import android.os.Build;import android.text.TextUtils;import android.util.Log;import android.view.accessibility.AccessibilityEvent;import android.view.accessibility.AccessibilityNodeInfo;import java.util.Collections;import java.util.List; import android.accessibilityservice.GestureDescription;import android.graphics.Path;import android.graphics.PixelFormat;import android.media.AudioManager;import android.view.Gravity;import android.view.LayoutInflater;import android.view.View;import android.view.WindowManager; import android.widget.Button;import android.widget.FrameLayout; import java.util.ArrayDeque;import java.util.Deque; /*Draft java code by "Lazarus Android Module Wizard" [7/14/2020 16:47:51]*//*https://github.com/jmpessoa/lazandroid modulewizard*//*jControl LAMW template*/ // ref. https://stackoverflow.com/questions/22057625/prevent-ussd-dialog-and- read-ussd-response// ref. http://habrahabr.ru/post/234425/ public class jUSSDService extends AccessibilityService { private long pascalObj = 0; //Pascal Object private Controls controls = null; //Java/Pascal [events] Interface ... private Context context = null; /*.*/public static String TAG = jUSSDService.class.getSimpleName(); //GUIDELINE: please, preferentially, init all yours params names with "_", ex: int _flag, String _hello ... /*.*/public jUSSDService() { //super(); } public jUSSDService(Controls _ctrls, long _Self) { //Add more others news "_xxx" params if needed! //super(_ctrls.activity); context = _ctrls.activity; pascalObj = _Self; controls = _ctrls; } private String processUSSDText(List eventText) { for (CharSequence s : eventText) { String text = String.valueOf(s); // Return text if text is the expected ussd response if( true ) { return text; } } return null; } public void logNodeHeirarchy(AccessibilityNodeInfo nodeInfo, int depth) { if (nodeInfo == null) return; String logString = ""; for (int i = 0; i < depth; ++i) { logString += " "; } logString += "Text: " + nodeInfo.getText() + " " + " Content- Description: " + nodeInfo.getContentDescription(); // Log.v("Meu", logString); Intent intent = new Intent("org.lamw.action.USSDService"); intent.putExtra("message",logString ); //text sendBroadcast(intent); for (int i = 0; i < nodeInfo.getChildCount(); ++i) { logNodeHeirarchy(nodeInfo.getChild(i), depth + 1); } } @Override /*.*/public void onAccessibilityEvent(AccessibilityEvent event) { //Log.d(TAG, "onAccessibilityEvent"); AccessibilityNodeInfo source = event.getSource(); if (source == null) { return;} if (source.getPackageName().equals("com.android.phone")) { /* if (event.getEventType() == AccessibilityEvent.TYPE_WINDOW_STATE_CHANGED && !event.getClassName().equals("android.app.AlertDialog")) { // android.app.AlertDialog is the standard but not for all phones */ if (event.getEventType() == AccessibilityEvent.TYPE_WINDOW_STATE_CHANGED && !String.valueOf(event.getClassName()).contains("AlertDialog")) { return; } if(event.getEventType() == AccessibilityEvent.TYPE_WINDOW_CONTENT_CHANGED && (source == null || !source.getClassName().equals("android.widget.TextView"))) { return; } if(event.getEventType() == AccessibilityEvent.TYPE_WINDOW_CONTENT_CHANGED && TextUtils.isEmpty(source.getText())) { return; } List eventText; if(event.getEventType() == AccessibilityEvent.TYPE_WINDOW_STATE_CHANGED) { eventText = event.getText(); } else { eventText = Collections.singletonList(source.getText()); } String text = processUSSDText(eventText); if( TextUtils.isEmpty(text) ) return; if(Build.VERSION.SDK_INT >= 16) { //need API >= 16! //[ifdef_api16up] performGlobalAction(GLOBAL_ACTION_BACK); //api >= 16 (android >= 4.1) //[endif_api16up] } //Log.d(TAG, text); //Handle USSD response here Intent intent = new Intent("org.lamw.action.USSDService"); intent.putExtra("message", "USSD:"+text); // sendBroadcast(intent); return; } if (!source.getPackageName().equals("com.sec.android.app.launcher")) { if (event.getEventType()==AccessibilityEvent.TYPE_WINDO W_CONTENT_CHANGED){ Intent intent = new Intent("org.lamw.action.USSDService"); intent.putExtra("message","Package:"+source.getPackageName() ); //text sendBroadcast(intent); AccessibilityNodeInfo currentNode=getRootInActiveWindow(); if (currentNode!=null) { logNodeHeirarchy(currentNode, 0); } } } } @Override /*.*/public void onInterrupt() { // } @Override protected void onServiceConnected() { super.onServiceConnected(); AccessibilityServiceInfo info = new AccessibilityServiceInfo(); info.flags = AccessibilityServiceInfo.DEFAULT; info.packageNames = null; info.eventTypes = AccessibilityEvent.TYPE_WINDOW_STATE_CHANGED | AccessibilityEvent.TYPE_WINDOW_CONTENT_CHANGED; info.feedbackType = AccessibilityServiceInfo.FEEDBACK_GENERIC; setServiceInfo(info); } public void jFree() { //free local objects... } //write others [public] methods code here...... //GUIDELINE: please, preferentially, init all yours params names with "_", ex: int _flag, String _hello ... //Method to start the service public void Start() { //Explicit intents explicitly define the component which should be called by the Android system, //by using the Java class as identifier //Create an intent for a specific component. // Intent intent = new Intent(controls.activity, jUSSDService.class); // controls.activity.startService(intent); Intent intent = new Intent(Settings.ACTION_ACCESSIBILITY_SETTINGS); controls.activity.startActivity(intent); } public void Stop() { controls.activity.stopService(new Intent(controls.activity,jUSSDService.class)); } } Important: In the Original Demo I added Uses strutils. AND : Code: Pascal [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i if findpart('USSD:',USSDResponse) >0 then ShowMessage(USSDResponse); Code: Pascal [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i procedure TAndroidModule1.jBroadcastReceiver1Receiver(Sender: TObject; intent: jObject);var USSDResponse: string; action: string;begin action:= jIntentManager1.GetAction(intent); if action = 'org.lamw.action.USSDService' then begin USSDResponse:= jIntentManager1.GetExtraString(intent, 'message'); if findpart('USSD:',USSDResponse) >0 then ShowMessage(USSDResponse); end;end; And XML Code: XML [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i

note: I kept the original code untouched, except in START() and in "USSD:" in the USSD response . The rest is add... There is an unnecessary line : 149. While I was testing found it convenient not to receive messages from the Launcher. But it must be excluded. ---------------------------------------------------------------------- Guser979 · 2021-09-23 12:00 A very simple usage example... Read twitters, including replies... Obviously, it's not limited to that. Usage depends on need and imagination. But a challenge for those who have time, being a programmer or not (like me).Having heard the tweets presented, it would be interesting to refresh the screen to continue reading while you're doing something else and don't want to or can't pick up the phone. Well, we would have to produce a gesture. In this case, Swipe Down. We will try? I'll. But if you can solve the challenge, share your (tested) solution. ---------------------------------------------------------------------- jmpessoa · 2021-09-23 20:53 QuoteMy solution is not that of a programmer, but it works... hehe Nice! Please, put your code and demo in some "open drive" ... I will commit your improvements to LAMW github! Thank you! ---------------------------------------------------------------------- Guser979 · 2021-09-23 23:03 Yes I'll do it. But it's sad to realize that so few programmers collaborate with this tool. It's light, practical and functional. It could be at a much higher level than it already is. I usually search for topics and with the word "Accessibility" the results are almost 0. Pure waste. But let's go ahead... ---------------------------------------------------------------------- Guser979 · 2021-09-24 20:48 Done https://drive.google.com/file/d/1v6DWeUC7BliljzRj5qS_wXgO8LD3k- Gf/view?usp=sharing ---------------------------------------------------------------------- jmpessoa · 2021-09-25 23:11 Done! Thank you! ---------------------------------------------------------------------- Guser979 · 2021-09-26 14:29 Thank you Master. Well.. Continuing with the challenge I set myself: The code for Swipe is tested Code: Java [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i public void Swipe() { if (path == null) { path = new Path(); path.moveTo(513,400);//// only y change (values x and y : your device) path.lineTo(513,1000);//// only y change (values x and y : your device) } if (builder == null) builder = new GestureDescription.Builder(); if (strokeDescription == null) { strokeDescription = new GestureDescription.StrokeDescription(path, 0, 100); builder.addStroke(strokeDescription); } if (gestureResultCallback == null) gestureResultCallback = new GestureResultCallback() { @Override public void onCompleted(GestureDescription gestureDescription) { } @Override public void onCancelled(GestureDescription gestureDescription) { super.onCancelled(gestureDescription); } }; if (gestureDescription == null) gestureDescription = builder.build(); dispatchGesture(gestureDescription, gestureResultCallback, null); } But now it is necessary to program the right moment to perform the gesture. In other words: exactly when jTextToSpeech finishes reading the twetts. There are several possible solutions. I'm testing. ---------------------------------------------------------------------- Guser979 · 2021-10-10 02:02 https://drive.google.com/file/d/1a6X7vHYBhf0J8LZnJfxEwu8OlOI30jjh/view ?usp=sharing Not to leave without conclusion, here it is. Goal : Read tweets using tts and refresh the screen by performing a Swipe gesture after reading. The java service Ussd was used as a basis. I added tts (in the same java unit) to it, along with the code for swipe. That is all. "Jussd Service" can now perform gestures in chosen apps, can speak and present the content of chosen screen. I think accessibility is being put to good use now. The file contains only unit.pas and Jussdservice.java with the modifications. Therefore, it is for those who have some knowledge of how LAMW works. Used : Jwindowmanager jBroadcastReceiver jUSSDService jPreferences jListView I also finished a minimal version of Synapse on Desktop which function is just to receive voice commands from android (using jTCPSocketClient and jCustomSpeechToText only) and perform any programmable action: Open programs, maximize minimize etc any window. Position windows. Change windows size. Submit any key combinations. Write texts.Position and click with the mouse anywhere on the screen, etc. I'll make it available soon. ---------------------------------------------------------------------- jmpessoa · 2021-10-10 02:52 Hi, Guser979! What about an "AppWindowManagerDemo2" or "AppUSSDServiceDemo2" ? Then I can try unify/merge and improve the LAMW "jUSSDService.java" ---------------------------------------------------------------------- Guser979 · 2021-10-10 02:54 Okay, master. Edit : I have been facing some health issues. Once I'm fine, I'll continue exploring this fantastic LAMW. I made some modifications before that and will share for those who want to use, modify, improve, etc. To be useful for more people, in the input screen you need at least a package name, the Swipe gesture coordinates, a timer to ensure there is always update( in the case of Twitter) and etc. Hope to come back soon to explore LAMW. https://drive.google.com/file/d/1raGd9kxDh_tRNNRjkQ5oniRA5AVAC6_q/view ?usp=sharing