yetangitu/ghidra
1
0
Fork 0
mirror of https://github.com/NationalSecurityAgency/ghidra.git synced 2025-10-04 10:19:23 +02:00
Code Issues Releases Wiki Activity

GP-4659: Renaming "Python" module to "Jython"

Browse source
This commit is contained in:
Ryan Kurtz 2024-06-03 11:29:56 -04:00
parent 9840eee937
commit fb1f725f5b
46 changed files with 285 additions and 284 deletions
Download patch file Download diff file Expand all files Collapse all files

9
Ghidra/Features/Jython/src/main/help/help/TOC_Source.xml Normal file
View file

@ -0,0 +1,9 @@
<?xml version='1.0' encoding='ISO-8859-1' ?>
<tocroot>
<tocref id="Ghidra Functionality">
<tocref id="Scripting">
<tocdef id="Jython Interpreter" sortgroup="z" text="Jython Interpreter" target="help/topics/Jython/interpreter.html" />
</tocref>
</tocref>
</tocroot>

171
Ghidra/Features/Jython/src/main/help/help/topics/Jython/interpreter.html Normal file
View file

@ -0,0 +1,171 @@
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN">
<HTML>
<HEAD>
<TITLE>Jython Interpreter</TITLE>
<LINK rel="stylesheet" type="text/css" href="help/shared/DefaultStyle.css">
</HEAD>
<BODY lang="EN-US">
<H1><A name="Jython"></A>Jython Interpreter</H1>
<P>
The Ghidra <I>Jython Interpreter</I> provides a full general-purpose Jython interactive shell
and allows you to interact with your current Ghidra session by exposing Ghidra's powerful Java
API through the magic of Jython.
</P>
<H2>Environment</H2>
<BLOCKQUOTE>
<P>
The Ghidra <I>Jython Interpreter</I> is configured to run in a similar context as a Ghidra
script. Therefore, you immediately have access to variables such as <TT>currentProgram</TT>,
<TT>currentSelection</TT>, <TT>currentAddress</TT>, etc without needing to import them.
These variables exist as Java objects behind the scenes, but Jython allows you to interact with
them through a Python interface, which is similar to Java in some ways.
</P>
<P>
As in Java, classes outside of your current package/module need to be explicitly imported.
For example, consider the following code snippet:
</P>
<BR>
<PRE>
<FONT COLOR="GREEN"># Get a data type from the user</FONT>
tool = state.getTool()
dtm = currentProgram.getDataTypeManager()
from ghidra.app.util.datatype import DataTypeSelectionDialog
from ghidra.util.data.DataTypeParser import AllowedDataTypes
selectionDialog = DataTypeSelectionDialog(tool, dtm, -1, AllowedDataTypes.FIXED_LENGTH)
tool.showDialog(selectionDialog)
dataType = selectionDialog.getUserChosenDataType()
if dataType != None: print "Chosen data type: " + str(dataType)
</PRE>
<P>
<TT>currentProgram</TT> and <TT>state</TT> are defined within the Ghidra scripting class
hierarchy, so nothing has to be explicitly imported before they can be used. However, because
the <TT>DataTypeSelectionDialog</TT> class and <TT>AllowedDataType</TT> enum reside in
different packages, they must be explicitly imported. Failure to do so will result in a
Jython <TT><FONT COLOR="RED">NameError</FONT></TT>.
</P>
</BLOCKQUOTE>
<H2><A name="Clear_Interpreter"></A>Clear <IMG border="0" src="images/erase16.png"></H2>
<BLOCKQUOTE>
<P>
This command clears the interpreter's display. Its effect is purely visual.
It does not affect the state of the interpreter in any way.
</P>
</BLOCKQUOTE>
<H2><A name="Interrupt_Interpreter"></A>Interrupt <IMG border="0" src="images/dialog-cancel.png"></H2>
<BLOCKQUOTE>
<P>
This command issues a keyboard interrupt to the interpreter, which can be used to interrupt
long running commands or loops.
</P>
</BLOCKQUOTE>
<H2><A name="Reset_Interpreter"></A>Reset <IMG border="0" src="images/reload3.png"></H2>
<BLOCKQUOTE>
<P>
This command resets the interpreter, which clears the display and resets all state.
</P>
</BLOCKQUOTE>
<H2>Keybindings</H2>
<BLOCKQUOTE>
<P>
The Ghidra <I>Jython Interpreter</I> supports the following hard-coded keybindings:
<UL>
<LI><B>(up):</B>&nbsp;&nbsp;Move backward in command stack</LI>
<LI><B>(down):</B>&nbsp;&nbsp;Move forward in command stack</LI>
<LI><B>TAB:</B>&nbsp;&nbsp;Show code completion window</LI>
</UL>
<P>
With the code completion window open:
<UL>
<LI><B>TAB:</B>&nbsp;&nbsp;Insert currently-selected code completion (if no completion selected, select the first available)</LI>
<LI><B>ENTER:</B>&nbsp;&nbsp;Insert selected completion (if any) and close the completion window</LI>
<LI><B>(up):</B>&nbsp;&nbsp;Select previous code completion</LI>
<LI><B>(down):</B>&nbsp;&nbsp;Select next code completion</LI>
<LI><B>ESC:</B>&nbsp;&nbsp;Hide code completion window</LI>
</UL>
</P>
</BLOCKQUOTE>
<H2>Copy/Paste</H2>
<BLOCKQUOTE>
<P>
Copy and paste from within the Ghidra <I>Jython Interpreter</I> should work as expected for
your given environment:
<UL>
<LI><B>Windows:</B>&nbsp;&nbsp;CTRL+C / CTRL+V</LI>
<LI><B>Linux:</B>&nbsp;&nbsp;CTRL+C / CTRL+V</LI>
<LI><B>OS X:</B>&nbsp;&nbsp;COMMAND+C / COMMAND+V</LI>
</UL>
</P>
</BLOCKQUOTE>
<H2>API Documentation</H2>
<BLOCKQUOTE>
<P>
The built-in <TT>help()</TT> Jython function has been altered by the Ghidra <I>Jython Interpreter</I>
to add support for displaying Ghidra's Javadoc (where available) for a given Ghidra class, method,
or variable. For example, to see Ghidra's Javadoc on the <TT>state</TT> variable, simply do:
<PRE>
>>> help(state)
#####################################################
class ghidra.app.script.GhidraState
extends java.lang.Object
Represents the current state of a Ghidra tool
#####################################################
PluginTool getTool()
Returns the current tool.
@return ghidra.framework.plugintool.PluginTool: the current tool
-----------------------------------------------------
Project getProject()
Returns the current project.
@return ghidra.framework.model.Project: the current project
-----------------------------------------------------
...
...
...
</PRE>
<P>
Calling help() with no arguments will show the Javadoc for the GhidraScript class.
</P>
<P>
<B>Note:</B> It may be necessary to import a Ghidra class before calling the built-in <TT>help()</TT>
Jython function on it. Failure to do so will result in a Jython <TT><FONT COLOR="RED">NameError</FONT></TT>.
</P>
</BLOCKQUOTE>
<H2>Additional Help</H2>
<BLOCKQUOTE>
<P>
For more information on the Jython environment, such as how to interact with Java objects
through a Python interface, please refer to Jython's free e-book which can be found on the
Internet at <I><B>www.jython.org/jythonbook/en/1.0/</B></I>
</P>
</BLOCKQUOTE>
<P align="left" class="providedbyplugin">Provided by: <I>JythonPlugin</I></P>
<P>&nbsp;</P>
<BR>
<BR>
<BR>
</BODY>
</HTML>

559
Ghidra/Features/Jython/src/main/java/ghidra/jython/GhidraJythonInterpreter.java Normal file
View file

@ -0,0 +1,559 @@
/* ###
* IP: GHIDRA
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package ghidra.jython;
import java.io.File;
import java.io.IOException;
import java.lang.reflect.*;
import java.net.InetAddress;
import java.net.Socket;
import java.util.*;
import org.python.core.*;
import org.python.util.InteractiveInterpreter;
import generic.jar.ResourceFile;
import ghidra.app.plugin.core.console.CodeCompletion;
import ghidra.app.script.GhidraScriptUtil;
import ghidra.util.Msg;
import ghidra.util.SystemUtilities;
import ghidra.util.exception.AssertException;
import ghidra.util.task.TaskMonitor;
import util.CollectionUtils;
/**
* A python interpreter meant for Ghidra's use. Each interpreter you get will have its own
* variable space so they should not interfere with each other.
* <p>
* There is no longer a way to reset an interpreter...it was too complicated to get right.
* Instead, you should {@link #cleanup()} your old interpreter and make a new one.
*/
public class GhidraJythonInterpreter extends InteractiveInterpreter {
private static boolean pythonInitialized;
private static List<PyString> defaultPythonPath;
private TraceFunction interruptTraceFunction;
private PyModule introspectModule;
private PyModule builtinModule;
private PyObject interrupt;
private boolean scriptMethodsInjected;
private boolean cleanedUp;
/**
* Gets a new GhidraPythonInterpreter instance.
*
* @return A new GhidraPythonInterpreter. Could be null if it failed to be created.
*/
public static GhidraJythonInterpreter get() {
// Initialize the python environment if necessary. Only needs to happen once.
if (!pythonInitialized) {
try {
// Setup python home directory
JythonUtils.setupJythonHomeDir();
// Setup python cache directory
JythonUtils.setupJythonCacheDir(TaskMonitor.DUMMY);
// Indicate that we've initialized the python environment, which should
// only happen once.
pythonInitialized = true;
}
catch (Exception e) {
Msg.showError(GhidraJythonInterpreter.class, null, "Python error",
"Problem getting Ghirda Python interpreter", e);
return null;
}
}
// Set up our default system state, including prompt styles.
PySystemState state = new PySystemState();
state.ps1 = new PyString(">>> ");
state.ps2 = new PyString("... ");
// Return a new instance of our interpreter
return new GhidraJythonInterpreter(state);
}
/**
* Creates a new Ghidra python interpreter object.
*
* @param state The initial system state of the interpreter.
*/
private GhidraJythonInterpreter(PySystemState state) {
super(null, state);
// Store the default python path in case we need to reset it later.
defaultPythonPath = new ArrayList<>();
for (Object object : systemState.path) {
defaultPythonPath.add(Py.newStringOrUnicode(object.toString()));
}
// Allow interruption of python code to occur when various code paths are
// encountered.
interruptTraceFunction = new InterruptTraceFunction();
// Setup __main__ module
PyModule mod = imp.addModule("__main__");
setLocals(mod.__dict__);
// Load site.py (standard Python practice).
// This will also load our sitecustomize.py module.
imp.load("site");
// Setup code completion module.
// Note that this is not exported to the global address space by default.
introspectModule = (PyModule) imp.load("jintrospect");
// Add __builtin__ module for code completion
builtinModule = (PyModule) imp.load("__builtin__");
}
/**
* Initializes/resets the python path to include all known Ghidra script paths.
*/
private void initializePythonPath() {
// Restore the python path back to default.
systemState.path.retainAll(defaultPythonPath);
// Add in Ghidra script source directories
for (ResourceFile resourceFile : GhidraScriptUtil.getEnabledScriptSourceDirectories()) {
systemState.path.append(Py.newStringOrUnicode(resourceFile.getFile(false).getAbsolutePath()));
}
for (ResourceFile resourceFile : GhidraScriptUtil.getExplodedCompiledSourceBundlePaths()) {
systemState.path.append(Py.newStringOrUnicode(resourceFile.getFile(false).getAbsolutePath()));
}
// Add in the PyDev remote debugger module
if (!SystemUtilities.isInDevelopmentMode()) {
File pyDevSrcDir = PyDevUtils.getPyDevSrcDir();
if (pyDevSrcDir != null) {
systemState.path.append(Py.newStringOrUnicode(pyDevSrcDir.getAbsolutePath()));
}
}
}
/**
* Pushes (executes) a line of Python to the interpreter.
*
* @param line the line of Python to push to the interpreter
* @param script a PythonScript from which we load state (or null)
* @return true if more input is needed before execution can occur
* @throws PyException if an unhandled exception occurred while executing the line of python
* @throws IllegalStateException if this interpreter has been cleaned up.
*/
public synchronized boolean push(String line, JythonScript script)
throws PyException, IllegalStateException {
if (cleanedUp) {
throw new IllegalStateException(
"Ghidra python interpreter has already been cleaned up.");
}
initializePythonPath();
injectScriptHierarchy(script);
if (buffer.length() > 0) {
buffer.append("\n");
}
buffer.append(line);
Py.getThreadState().tracefunc = interruptTraceFunction;
Py.getSystemState().stderr = getSystemState().stderr; // needed to properly display SyntaxError
boolean more;
try {
more = runsource(buffer.toString(), "python");
getSystemState().stderr.invoke("flush");
if (!more) {
resetbuffer();
}
}
catch (PyException pye) {
resetbuffer();
throw pye;
}
return more;
}
/**
* Execute a python file using this interpreter.
*
* @param file The python file to execute.
* @param script A PythonScript from which we load state (or null).
* @throws IllegalStateException if this interpreter has been cleaned up.
*/
public synchronized void execFile(ResourceFile file, JythonScript script)
throws IllegalStateException {
if (cleanedUp) {
throw new IllegalStateException(
"Ghidra python interpreter has already been cleaned up.");
}
initializePythonPath();
injectScriptHierarchy(script);
Py.getThreadState().tracefunc = interruptTraceFunction;
// The Python import system sets the __file__ attribute to the file it's executing
setVariable("__file__", new PyString(file.getAbsolutePath()));
// If the remote python debugger is alive, initialize it by calling settrace()
if (!SystemUtilities.isInDevelopmentMode() && !SystemUtilities.isInHeadlessMode()) {
if (PyDevUtils.getPyDevSrcDir() != null) {
try {
InetAddress localhost = InetAddress.getLocalHost();
new Socket(localhost, PyDevUtils.PYDEV_REMOTE_DEBUGGER_PORT).close();
Msg.info(this, "Python debugger found");
StringBuilder dbgCmds = new StringBuilder();
dbgCmds.append("import pydevd;");
dbgCmds.append("pydevd.threadingCurrentThread().__pydevd_main_thread = True;");
dbgCmds.append("pydevd.settrace(host=\"" + localhost.getHostName() +
"\", port=" + PyDevUtils.PYDEV_REMOTE_DEBUGGER_PORT + ", suspend=False);");
exec(dbgCmds.toString());
Msg.info(this, "Connected to a python debugger.");
}
catch (IOException e) {
Msg.info(this, "Not connected to a python debugger.");
}
}
}
// Run python file
execfile(file.getAbsolutePath());
}
@Override
public synchronized void cleanup() {
super.cleanup();
cleanedUp = true;
}
/**
* Prints the given string to the interpreter's error stream with a newline
* appended.
*
* @param str The string to print.
*/
void printErr(String str) {
try {
getSystemState().stderr.invoke("write", new PyString(str + "\n"));
getSystemState().stderr.invoke("flush");
}
catch (PyException e) {
// if the python interp state's stdin/stdout/stderr is messed up, it can throw an error
Msg.error(this, "Failed to write to stderr", e);
}
}
/**
* Gets the interpreter's primary prompt.
*
* @return The interpreter's primary prompt.
*/
synchronized String getPrimaryPrompt() {
return getSystemState().ps1.toString();
}
/**
* Gets the interprester's secondary prompt.
*
* @return The interpreter's secondary prompt.
*/
synchronized String getSecondaryPrompt() {
return getSystemState().ps2.toString();
}
/**
* Handle a KeyboardInterrupt.
* <p>
* This will attempt to interrupt the interpreter if it is running. There are
* two types of things this interrupt will work on:
* <p>
* 1: A batched series of python commands (such as a loop). This works by setting
* our interrupt flag that is checked by our {@link InterruptTraceFunction} when
* various trace events happen.
* <p>
* 2: A sleeping or otherwise interruptible python command. Since jython is all
* java under the hood, a sleep is really just a {@link Thread#sleep}, which we can
* kick with a {@link Thread#interrupt()}.
* <p>
* If another type of thing is taking a really long time, this interrupt will fail.
*
* @param pythonThread The Python Thread we need to interrupt.
*/
void interrupt(Thread pythonThread) {
final long INTERRUPT_TIMEOUT = 5000;
if ((pythonThread != null) && pythonThread.isAlive()) {
// Set trace interrupt flag
interrupt = Py.KeyboardInterrupt;
// Wake potentially sleeping python command
pythonThread.interrupt();
try {
pythonThread.join(INTERRUPT_TIMEOUT);
if (pythonThread.isAlive()) {
printErr("Cannot interrupt running command");
}
}
catch (InterruptedException e) {
// Nothing to do
}
interrupt = null;
}
else {
printErr("KeyboardInterrupt");
}
resetbuffer();
}
/**
* Injects all of the accessible fields and methods found in the PythonScript class hierarchy into
* the given interpreter's Python address space.
*
* @param script The script whose class hierarchy is to be used for injection.
*/
void injectScriptHierarchy(JythonScript script) {
if (script == null) {
return;
}
// Inject 'this'
setVariable("this", script);
// Loop though the script class hierarchy
for (Class<?> scriptClass = script.getClass(); scriptClass != Object.class; scriptClass =
scriptClass.getSuperclass()) {
// Add public and protected fields
for (Field field : scriptClass.getDeclaredFields()) {
if (Modifier.isPublic(field.getModifiers()) ||
Modifier.isProtected(field.getModifiers())) {
try {
field.setAccessible(true);
setVariable(field.getName(), field.get(script));
}
catch (IllegalAccessException iae) {
throw new AssertException("Unexpected security manager being used!");
}
}
}
// Add public methods (only once). Ignore inner classes.
//
// NOTE: We currently do not have a way to safely add protected methods. Disabling
// python.security.respectJavaAccessibility and adding in protected methods in the below
// loop caused an InaccessibleObjectException for some users (relating to core Java
// modules, not the GhidraScript class hierarchy).
if (!scriptMethodsInjected) {
for (Method method : scriptClass.getDeclaredMethods()) {
if (!method.getName().contains("$") &&
Modifier.isPublic(method.getModifiers())) {
method.setAccessible(true);
setMethod(script, method);
}
}
}
}
scriptMethodsInjected = true;
}
/**
* Safely sets a variable in the interpreter's namespace. This first checks to
* make sure that we are not overriding a builtin Python symbol.
*
* @param varName The name of variable.
* @param obj The value of the variable.
* @return True if the variable was set; false if it already existed and wasn't set.
*/
private boolean setVariable(String varName, Object obj) {
if (builtinModule.__findattr__(varName) == null) {
set(varName, obj);
return true;
}
return false;
}
/**
* Sets a bound (callback/function pointer) method as a local variable in the interpreter.
*
* @param obj A Java object that contains the method to bind.
* @param method The method from the object to bind.
* @return True if the method was set; false if it already existed and wasn't set.
*/
private boolean setMethod(Object obj, Method method) {
String methodName = method.getName();
// First, check to make sure we're not shadowing any internal Python keywords/functions/etc
if (builtinModule.__findattr__(methodName) != null) {
return false;
}
// OK, we're safe to set it
PyObject pyObj = get(methodName);
if ((null == pyObj) || (pyObj instanceof PyNone)) {
// This is the first method of this name that we are adding. Create a new bound PyMethod
// to bind the Java method to the Java object in the Python world.
set(methodName, new PyMethod(new PyReflectedFunction(method), Py.java2py(obj),
Py.java2py(obj.getClass())));
}
else if (pyObj instanceof PyMethod) {
// Another method of this name has already been added. Add it to the list of possibilities
// (different arguments to methods on the same Object). But first, we must do some sanity
// checks.
PyMethod pyMethod = (PyMethod) pyObj;
if ((pyMethod.__self__._is(Py.java2py(obj))) != Py.True) {
Msg.error(this,
"Method " + methodName + " of " + obj + " attempting to shadow method " +
pyMethod.__func__ + " of " + pyMethod.__self__);
return false;
}
if (!(pyMethod.__func__ instanceof PyReflectedFunction)) {
Msg.error(this, "For addition of method " + methodName + " of " + obj +
", cannot mix with non Java function " + pyMethod.__func__);
return false;
}
((PyReflectedFunction) pyMethod.__func__).addMethod(method);
}
return true;
}
/**
* Returns the possible command completions for a command.
*
* @param cmd The command line.
* @param includeBuiltins True if we should include python built-ins; otherwise, false.
* @param caretPos The position of the caret in the input string 'cmd'
* @return A list of possible command completions. Could be empty if there aren't any.
* @see JythonPlugin#getCompletions
*/
List<CodeCompletion> getCommandCompletions(String cmd, boolean includeBuiltins, int caretPos) {
// At this point the caret is assumed to be positioned right after the value we need to
// complete (example: "[complete.Me<caret>, rest, code]"). To make the completion work
// in our case, it's sufficient (albeit naive) to just remove the text on the right side
// of our caret. The later code (on the python's side) will parse the rest properly
// and will generate the completions.
cmd = cmd.substring(0, caretPos);
if ((cmd.length() > 0) && (cmd.charAt(cmd.length() - 1) == '(')) {
return getMethodCommandCompletions(cmd);
}
return getPropertyCommandCompletions(cmd, includeBuiltins);
}
/**
* Returns method documentation for the current command.
*
* @param cmd the current command
* @return method documentation for the current command
*/
private List<CodeCompletion> getMethodCommandCompletions(String cmd) {
List<CodeCompletion> completion_list = new ArrayList<>();
try {
PyObject getCallTipJava = introspectModule.__findattr__("getCallTipJava");
PyString command = new PyString(cmd);
PyObject locals = getLocals();
// Return value is (name, argspec, tip_text)
ListIterator<?> iter =
((List<?>) getCallTipJava.__call__(command, locals)).listIterator();
while (iter.hasNext()) {
String completion_portion = iter.next().toString();
if (!completion_portion.equals("")) {
String[] substrings = completion_portion.split("\n");
for (String substring : substrings) {
completion_list.add(new CodeCompletion(substring, null, null));
}
}
}
}
catch (Exception e) {
Msg.error(this, "Unexpected Exception: " + e.getMessage(), e);
}
return completion_list;
}
/**
* Returns a Map of property-&gt;string_substitution pairs.
*
* @param cmd current command
* @param includeBuiltins True if we should include python built-ins; otherwise, false.
* @return A list of possible command completions. Could be empty if there aren't any.
*/
private List<CodeCompletion> getPropertyCommandCompletions(String cmd,
boolean includeBuiltins) {
try {
PyObject getAutoCompleteList = introspectModule.__findattr__("getAutoCompleteList");
PyString command = new PyString(cmd);
PyStringMap locals = ((PyStringMap) getLocals()).copy();
if (includeBuiltins) {
// Add in the __builtin__ module's contents for the search
locals.update(builtinModule.__dict__);
}
List<?> list = (List<?>) getAutoCompleteList.__call__(command, locals);
return CollectionUtils.asList(list, CodeCompletion.class);
}
catch (Exception e) {
Msg.error(this, "Unexpected Exception: " + e.getMessage(), e);
return Collections.emptyList();
}
}
/**
* Custom trace function that allows interruption of python code to occur when various code
* paths are encountered.
*/
class InterruptTraceFunction extends TraceFunction {
private void checkInterrupt() {
if (interrupt != null) {
throw Py.makeException(interrupt);
}
}
@Override
public TraceFunction traceCall(PyFrame frame) {
checkInterrupt();
return this;
}
@Override
public TraceFunction traceReturn(PyFrame frame, PyObject ret) {
checkInterrupt();
return this;
}
@Override
public TraceFunction traceLine(PyFrame frame, int line) {
checkInterrupt();
return this;
}
@Override
public TraceFunction traceException(PyFrame frame, PyException exc) {
checkInterrupt();
return this;
}
}
}

301
Ghidra/Features/Jython/src/main/java/ghidra/jython/JythonCodeCompletionFactory.java Normal file
View file

@ -0,0 +1,301 @@
/* ###
* IP: GHIDRA
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package ghidra.jython;
import java.awt.Color;
import java.lang.reflect.Method;
import java.util.*;
import javax.swing.JComponent;
import org.python.core.PyInstance;
import org.python.core.PyObject;
import docking.widgets.label.GDLabel;
import generic.theme.GColor;
import ghidra.app.plugin.core.console.CodeCompletion;
import ghidra.framework.options.Options;
import ghidra.util.Msg;
/**
* Generates CodeCompletions from Jython objects.
*
*
*
*/
public class JythonCodeCompletionFactory {
private static List<Class<?>> classes = new ArrayList<>();
private static Map<Class<?>, Color> classToColorMap = new HashMap<>();
/* necessary because we only want to show the user the simple class name
* Well, that, and the Options.DELIMITER is a '.' which totally messes
* things up.
*/
private static Map<String, Class<?>> simpleNameToClass = new HashMap<>();
private static Map<Class<?>, String> classDescription = new HashMap<>();
public static final String COMPLETION_LABEL = "Code Completion Colors";
/* package-level accessibility so that JythonPlugin can tell this is
* our option
*/
final static String INCLUDE_TYPES_LABEL = "Include type names in code completion popup?";
private final static String INCLUDE_TYPES_DESCRIPTION =
"Whether or not to include the type names (classes) of the possible " +
"completions in the code completion window. The class name will be " +
"parenthesized after the completion.";
private final static boolean INCLUDE_TYPES_DEFAULT = true;
private static boolean includeTypes = INCLUDE_TYPES_DEFAULT;
//@formatter:off
public static final Color NULL_COLOR = new GColor("color.fg.plugin.jython.syntax.null");
public static final Color FUNCTION_COLOR = new GColor("color.fg.plugin.jython.syntax.function");
public static final Color PACKAGE_COLOR = new GColor("color.fg.plugin.jython.syntax.package");
public static final Color CLASS_COLOR = new GColor("color.fg.plugin.jython.syntax.class");
public static final Color METHOD_COLOR = new GColor("color.fg.plugin.jython.syntax.method");
/* anonymous code chunks */
public static final Color CODE_COLOR = new GColor("color.fg.plugin.jython.syntax.code");
public static final Color INSTANCE_COLOR = new GColor("color.fg.plugin.jython.syntax.instance");
public static final Color SEQUENCE_COLOR = new GColor("color.fg.plugin.jython.syntax.sequence");
public static final Color MAP_COLOR = new GColor("color.fg.plugin.jython.syntax.map");
public static final Color NUMBER_COLOR = new GColor("color.fg.plugin.jython.syntax.number");
/* for weird Jython-specific stuff */
public static final Color SPECIAL_COLOR = new GColor("color.fg.plugin.jython.syntax.special");
//@formatter:on
static {
/* Order matters! This is the order in which classes are checked for
* coloring.
*/
setupClass("org.python.core.PyNone", NULL_COLOR, "'None' (null) Objects");
setupClass("org.python.core.PyReflectedFunction", FUNCTION_COLOR,
"Python functions written in Java");
/* changed for Jython 2.5 */
// setupClass("org.python.core.BuiltinFunctions", FUNCTION_COLOR,
// "Python's built-in functions collection (note that many are " +
// "re-implemented in Java)");
setupClass("org.python.core.__builtin__", FUNCTION_COLOR,
"Python's built-in functions collection (note that many are " +
"re-implemented in Java)");
setupClass("org.python.core.PyFunction", FUNCTION_COLOR, "functions written in Python");
setupClass("org.python.core.PyMethodDescr", FUNCTION_COLOR,
"unbound Python builtin instance methods (they take an " +
"Object as the first argument)");
setupClass("org.python.core.PyJavaPackage", PACKAGE_COLOR, "Java packages");
setupClass("org.python.core.PyModule", PACKAGE_COLOR, "Python modules");
/* Even though the latter is a subclass of the former, this allows
* the user to differentiate visually Java classes from Python classes
* if they so wish. But we don't do this by default.
*/
/* changed for Jython 2.5 */
// setupClass("org.python.core.PyJavaClass", CLASS_COLOR,
// "Java classes");
setupClass("org.python.core.PyJavaType", CLASS_COLOR, "Java classes");
setupClass("org.python.core.PyClass", CLASS_COLOR, "Python classes");
setupClass("org.python.core.PyType", CLASS_COLOR, "core Python types");
setupClass("org.python.core.PyMethod", METHOD_COLOR, "methods");
setupClass("org.python.core.PyBuiltinFunction", METHOD_COLOR,
"core Python methods, often inherited from Python's Object " +
"(overriding these methods is very powerful)");
setupClass("org.python.core.PySequence", SEQUENCE_COLOR,
"iterable sequences, including arrays, list, and strings");
setupClass("org.python.core.PyDictionary", MAP_COLOR, "arbitrary Python mapping type");
setupClass("org.python.core.PyStringMap", MAP_COLOR, "Python String->Object mapping type");
setupClass("org.python.core.PyInteger", NUMBER_COLOR, "integers");
setupClass("org.python.core.PyLong", NUMBER_COLOR, "long integers");
setupClass("org.python.core.PyFloat", NUMBER_COLOR, "floating-point (decimal) numbers");
setupClass("org.python.core.PyComplex", NUMBER_COLOR, "complex numbers");
setupClass("org.python.core.PyCompoundCallable", SPECIAL_COLOR,
"special Python properties for " +
"assigning Python functions as EventListeners on Java objects");
/* changed for Jython 2.5 */
setupClass("org.python.core.PyObjectDerived", INSTANCE_COLOR, "Java Objects");
setupClass("org.python.core.PyInstance", INSTANCE_COLOR, "Python Objects");
setupClass("org.python.core.PyCode", CODE_COLOR, "chunks of Python code");
}
/**
* Returns the actual class name for a Class.
*
* @param klass a Class
* @return The actual class name.
*/
private static String getSimpleName(Class<?> klass) {
return getSimpleName(klass.getName());
}
/**
* Returns the actual class name for a Class.
*
* @param className name of a Class
* @return The actual class name.
*/
private static String getSimpleName(String className) {
/* lastIndexOf returns -1 on not found, so this works whether or not
* a period is actually in className
*/
return className.substring(className.lastIndexOf('.') + 1);
}
/**
* Sets up a Class mapping.
*
* @param className Class name
* @param defaultColor default Color for this Class
* @param description description of the Class
*/
private static void setupClass(String className, Color defaultColor, String description) {
try {
Class<?> klass = Class.forName(className);
classes.add(klass);
classToColorMap.put(klass, defaultColor);
simpleNameToClass.put(getSimpleName(klass), klass);
classDescription.put(klass, description);
}
catch (ClassNotFoundException cnfe) {
Msg.debug(JythonCodeCompletionFactory.class, "Unable to find class: " + className,
cnfe);
}
}
/**
* Creates a new CodeCompletion from the given Jython objects.
*
* @param description description of the new CodeCompletion
* @param insertion what will be inserted to make the code complete
* @param pyObj a Jython Object
* @return A new CodeCompletion from the given Jython objects.
* @deprecated use {@link #newCodeCompletion(String, String, PyObject, String)} instead,
* it allows creation of substituting code completions
*/
@Deprecated
public static CodeCompletion newCodeCompletion(String description, String insertion,
PyObject pyObj) {
return newCodeCompletion(description, insertion, pyObj, "");
}
/**
* Creates a new CodeCompletion from the given Jython objects.
*
* @param description description of the new CodeCompletion
* @param insertion what will be inserted to make the code complete
* @param pyObj a Jython Object
* @param userInput a word we want to complete, can be an empty string.
* It's used to determine which part (if any) of the input should be
* removed before the insertion of the completion
* @return A new CodeCompletion from the given Jython objects.
*/
public static CodeCompletion newCodeCompletion(String description, String insertion,
PyObject pyObj, String userInput) {
JComponent comp = null;
if (pyObj != null) {
if (includeTypes) {
/* append the class name to the end of the description */
String className = getSimpleName(pyObj.getClass());
if (pyObj instanceof PyInstance) {
/* get the real class */
className = getSimpleName(((PyInstance) pyObj).instclass.__name__);
}
else if (className.startsWith("Py")) {
/* strip off the "Py" */
className = className.substring("Py".length());
}
description = description + " (" + className + ")";
}
comp = new GDLabel(description);
Iterator<Class<?>> iter = classes.iterator();
while (iter.hasNext()) {
Class<?> testClass = iter.next();
if (testClass.isInstance(pyObj)) {
comp.setForeground(classToColorMap.get(testClass));
break;
}
}
}
int charsToRemove = userInput.length();
return new CodeCompletion(description, insertion, comp, charsToRemove);
}
/**
* Sets up Jython code completion Options.
* @param plugin jython plugin as options owner
* @param options an Options handle
*/
public static void setupOptions(JythonPlugin plugin, Options options) {
includeTypes = options.getBoolean(INCLUDE_TYPES_LABEL, INCLUDE_TYPES_DEFAULT);
options.registerOption(INCLUDE_TYPES_LABEL, INCLUDE_TYPES_DEFAULT, null,
INCLUDE_TYPES_DESCRIPTION);
}
/**
* Handle an Option change.
*
* This is named slightly differently because it is a static method, not
* an instance method.
*
* By the time we get here, we assume that the Option changed is indeed
* ours.
*
* @param options the Options handle
* @param name name of the Option changed
* @param oldValue the old value
* @param newValue the new value
*/
public static void changeOptions(Options options, String name, Object oldValue,
Object newValue) {
if (name.equals(INCLUDE_TYPES_LABEL)) {
includeTypes = ((Boolean) newValue).booleanValue();
}
else {
Msg.error(JythonCodeCompletionFactory.class, "unknown option '" + name + "'");
}
}
/**
* Returns the Java __call__ methods declared for a Jython object.
*
* Some Jython "methods" in the new-style Jython objects are actually
* classes in and of themselves, re-implementing __call__ methods to
* tell us how to call them. This returns an array of those Methods
* (for code completion help).
*
* @param obj a PyObject
* @return the Java __call__ methods declared for the Jython object
*/
public static Object[] getCallMethods(PyObject obj) {
List<Method> callMethodList = new ArrayList<>();
Method[] declaredMethods = obj.getClass().getDeclaredMethods();
for (Method declaredMethod : declaredMethods) {
if (declaredMethod.getName().equals("__call__")) {
callMethodList.add(declaredMethod);
}
}
return callMethodList.toArray();
}
}

388
Ghidra/Features/Jython/src/main/java/ghidra/jython/JythonPlugin.java Normal file
View file

@ -0,0 +1,388 @@
/* ###
* IP: GHIDRA
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package ghidra.jython;
import java.awt.event.KeyEvent;
import java.io.*;
import java.util.List;
import javax.swing.Icon;
import org.python.core.PySystemState;
import docking.ActionContext;
import docking.DockingUtils;
import docking.action.*;
import generic.jar.ResourceFile;
import generic.theme.GIcon;
import ghidra.app.CorePluginPackage;
import ghidra.app.plugin.PluginCategoryNames;
import ghidra.app.plugin.ProgramPlugin;
import ghidra.app.plugin.core.console.CodeCompletion;
import ghidra.app.plugin.core.interpreter.*;
import ghidra.app.script.GhidraState;
import ghidra.framework.options.OptionsChangeListener;
import ghidra.framework.options.ToolOptions;
import ghidra.framework.plugintool.PluginInfo;
import ghidra.framework.plugintool.PluginTool;
import ghidra.framework.plugintool.util.PluginStatus;
import ghidra.util.HelpLocation;
import ghidra.util.task.*;
import resources.Icons;
/**
* This plugin provides the interactive Jython interpreter.
*/
//@formatter:off
@PluginInfo(
status = PluginStatus.RELEASED,
packageName = CorePluginPackage.NAME,
category = PluginCategoryNames.COMMON,
shortDescription = "Jython Interpreter",
description = "Provides an interactive Jython Interpreter that is tightly integrated with a loaded Ghidra program.",
servicesRequired = { InterpreterPanelService.class },
isSlowInstallation = true
)
//@formatter:on
public class JythonPlugin extends ProgramPlugin
implements InterpreterConnection, OptionsChangeListener {
private InterpreterConsole console;
private GhidraJythonInterpreter interpreter;
private JythonScript interactiveScript;
private TaskMonitor interactiveTaskMonitor;
private JythonPluginInputThread inputThread;
// Plugin options
private final static String INCLUDE_BUILTINS_LABEL = "Include \"builtins\" in code completion?";
private final static String INCLUDE_BUILTINS_DESCRIPTION =
"Whether or not to include Jython's built-in functions and properties in the pop-up code completion window.";
private final static boolean INCLUDE_BUILTINS_DEFAULT = true;
private static final Icon ICON = new GIcon("icon.plugin.jython");
private boolean includeBuiltins = INCLUDE_BUILTINS_DEFAULT;
/**
* Creates a new {@link JythonPlugin} object.
*
* @param tool The tool associated with this plugin.
*/
public JythonPlugin(PluginTool tool) {
super(tool);
}
/**
* Gets the plugin's interpreter console.
*
* @return The plugin's interpreter console.
*/
InterpreterConsole getConsole() {
return console;
}
/**
* Gets the plugin's Jython interpreter.
*
* @return The plugin's Jython interpreter. May be null.
*/
GhidraJythonInterpreter getInterpreter() {
return interpreter;
}
/**
* Gets the plugin's interactive script
*
* @return The plugin's interactive script.
*/
JythonScript getInteractiveScript() {
return interactiveScript;
}
/**
* Gets the plugin's interactive task monitor.
*
* @return The plugin's interactive task monitor.
*/
TaskMonitor getInteractiveTaskMonitor() {
return interactiveTaskMonitor;
}
@Override
protected void init() {
super.init();
console =
getTool().getService(InterpreterPanelService.class).createInterpreterPanel(this, false);
welcome();
console.addFirstActivationCallback(() -> resetInterpreter());
createActions();
}
/**
* Creates various actions for the plugin.
*/
private void createActions() {
// Interrupt Interpreter
DockingAction interruptAction = new DockingAction("Interrupt Interpreter", getName()) {
@Override
public void actionPerformed(ActionContext context) {
interrupt();
}
};
interruptAction.setDescription("Interrupt Interpreter");
interruptAction.setToolBarData(
new ToolBarData(Icons.NOT_ALLOWED_ICON, null));
interruptAction.setEnabled(true);
interruptAction.setKeyBindingData(
new KeyBindingData(KeyEvent.VK_I, DockingUtils.CONTROL_KEY_MODIFIER_MASK));
interruptAction.setHelpLocation(new HelpLocation(getTitle(), "Interrupt_Interpreter"));
console.addAction(interruptAction);
// Reset Interpreter
DockingAction resetAction = new DockingAction("Reset Interpreter", getName()) {
@Override
public void actionPerformed(ActionContext context) {
reset();
}
};
resetAction.setDescription("Reset Interpreter");
resetAction.setToolBarData(
new ToolBarData(Icons.REFRESH_ICON, null));
resetAction.setEnabled(true);
resetAction.setKeyBindingData(
new KeyBindingData(KeyEvent.VK_D, DockingUtils.CONTROL_KEY_MODIFIER_MASK));
resetAction.setHelpLocation(new HelpLocation(getTitle(), "Reset_Interpreter"));
console.addAction(resetAction);
}
/**
* Resets the interpreter to a new starting state. This is used when the plugin is first
* initialized, as well as when an existing interpreter receives a Jython exit command.
* We used to try to reset the same interpreter, but it was really hard to do that correctly
* so we now just create a brand new one.
* <p>
* NOTE: Loading Jython for the first time can be quite slow the first time, so we do this
* when the user wants to first interact with the interpreter (rather than when the plugin loads).
*/
private void resetInterpreter() {
TaskLauncher.launchModal("Resetting Jython...", () -> {
resetInterpreterInBackground();
});
}
// we expect this to be called from off the Swing thread
private void resetInterpreterInBackground() {
// Reset the interpreter by creating a new one. Clean up the old one if present.
if (interpreter == null) {
// Setup options
ToolOptions options = tool.getOptions("Jython");
includeBuiltins = options.getBoolean(INCLUDE_BUILTINS_LABEL, INCLUDE_BUILTINS_DEFAULT);
options.registerOption(INCLUDE_BUILTINS_LABEL, INCLUDE_BUILTINS_DEFAULT, null,
INCLUDE_BUILTINS_DESCRIPTION);
options.addOptionsChangeListener(this);
interpreter = GhidraJythonInterpreter.get();
// Setup code completion. This currently has to be done after the interpreter
// is created. Otherwise an exception will occur.
JythonCodeCompletionFactory.setupOptions(this, options);
}
else {
inputThread.shutdown();
inputThread = null;
interpreter.cleanup();
interpreter = GhidraJythonInterpreter.get();
}
// Reset the console.
console.clear();
console.setPrompt(interpreter.getPrimaryPrompt());
// Tie the interpreter's input/output to the plugin's console.
interpreter.setIn(console.getStdin());
interpreter.setOut(console.getStdOut());
interpreter.setErr(console.getStdErr());
// Print a welcome message.
welcome();
// Setup the JythonScript describing the state of the interactive prompt.
// This allows things like currentProgram and currentAddress to dynamically reflect
// what's happening in the listing. Injecting the script hierarchy early here allows
// code completion to work before commands are entered.
interactiveScript = new JythonScript();
interactiveScript.set(
new GhidraState(tool, tool.getProject(), getCurrentProgram(), getProgramLocation(),
getProgramSelection(), getProgramHighlight()),
interactiveTaskMonitor, new PrintWriter(getConsole().getStdOut()));
interpreter.injectScriptHierarchy(interactiveScript);
interactiveTaskMonitor = new JythonInteractiveTaskMonitor(console.getStdOut());
// Start the input thread that receives jython commands to execute.
inputThread = new JythonPluginInputThread(this);
inputThread.start();
}
/**
* Handle a change in one of our options.
*
* @param options the options handle
* @param optionName name of the option changed
* @param oldValue the old value
* @param newValue the new value
*/
@Override
public void optionsChanged(ToolOptions options, String optionName, Object oldValue,
Object newValue) {
if (optionName.startsWith(JythonCodeCompletionFactory.COMPLETION_LABEL)) {
JythonCodeCompletionFactory.changeOptions(options, optionName, oldValue, newValue);
}
else if (optionName.equals(JythonCodeCompletionFactory.INCLUDE_TYPES_LABEL)) {
JythonCodeCompletionFactory.changeOptions(options, optionName, oldValue, newValue);
}
else if (optionName.equals(INCLUDE_BUILTINS_LABEL)) {
includeBuiltins = ((Boolean) newValue).booleanValue();
}
}
/**
* Returns a list of possible command completion values.
*
* @param cmd current command line (without prompt)
* @return A list of possible command completion values. Could be empty if there aren't any.
*/
@Override
public List<CodeCompletion> getCompletions(String cmd) {
return getCompletions(cmd, cmd.length());
}
/**
* Returns a list of possible command completion values at the given position.
*
* @param cmd current command line (without prompt)
* @param caretPos The position of the caret in the input string 'cmd'
* @return A list of possible command completion values. Could be empty if there aren't any.
*/
@Override
public List<CodeCompletion> getCompletions(String cmd, int caretPos) {
// Refresh the environment
interactiveScript.setSourceFile(new ResourceFile(new File("jython")));
interactiveScript.set(
new GhidraState(tool, tool.getProject(), currentProgram, currentLocation,
currentSelection, currentHighlight),
interactiveTaskMonitor, console.getOutWriter());
return interpreter.getCommandCompletions(cmd, includeBuiltins, caretPos);
}
@Override
protected void dispose() {
// Do an interrupt in case there is a loop or something running
interrupt();
// Terminate the input thread
if (inputThread != null) {
inputThread.shutdown();
inputThread = null;
}
// Dispose of the console
if (console != null) {
console.dispose();
console = null;
}
// Cleanup the interpreter
if (interpreter != null) {
interpreter.cleanup();
interpreter = null;
}
super.dispose();
}
/**
* Interrupts what the interpreter is currently doing.
*/
public void interrupt() {
if (interpreter == null) {
return;
}
interpreter.interrupt(inputThread.getJythonPluginExecutionThread());
console.setPrompt(interpreter.getPrimaryPrompt());
}
/**
* Resets the interpreter's state.
*/
public void reset() {
// Do an interrupt in case there is a loop or something running
interrupt();
resetInterpreter();
}
@Override
public String getTitle() {
return "Jython";
}
@Override
public String toString() {
return getPluginDescription().getName();
}
@Override
public Icon getIcon() {
return ICON;
}
/**
* Prints a welcome message to the console.
*/
private void welcome() {
console.getOutWriter().println("Jython Interpreter for Ghidra");
console.getOutWriter().println("Based on Jython version " + PySystemState.version);
console.getOutWriter().println("Press 'F1' for usage instructions");
}
/**
* Support for cancelling execution using a TaskMonitor.
*/
class JythonInteractiveTaskMonitor extends TaskMonitorAdapter {
private PrintWriter output = null;
public JythonInteractiveTaskMonitor(PrintWriter stdOut) {
output = stdOut;
}
public JythonInteractiveTaskMonitor(OutputStream stdout) {
this(new PrintWriter(stdout));
}
@Override
public void setMessage(String message) {
output.println("<jython-interactive>: " + message);
}
}
}

97
Ghidra/Features/Jython/src/main/java/ghidra/jython/JythonPluginExecutionThread.java Normal file
View file

@ -0,0 +1,97 @@
/* ###
* IP: GHIDRA
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package ghidra.jython;
import java.io.File;
import java.io.PrintWriter;
import java.util.concurrent.atomic.AtomicBoolean;
import org.python.core.PyException;
import db.Transaction;
import generic.jar.ResourceFile;
import ghidra.app.script.GhidraState;
import ghidra.framework.plugintool.PluginTool;
import ghidra.program.model.listing.Program;
import ghidra.util.task.TaskMonitor;
/**
* Thread responsible for executing a jython command for the plugin.
*/
class JythonPluginExecutionThread extends Thread {
private JythonPlugin plugin;
private String cmd;
private AtomicBoolean moreInputWanted;
/**
* Creates a new jython plugin execution thread that executes the given command for the given
* plugin.
*
* @param plugin The jython plugin to execute the command for.
* @param cmd The jython command to execute.
* @param moreInputWanted Gets set to indicate that the executed command expects more input.
*/
JythonPluginExecutionThread(JythonPlugin plugin, String cmd, AtomicBoolean moreInputWanted) {
super("Jython plugin execution thread");
this.plugin = plugin;
this.cmd = cmd;
this.moreInputWanted = moreInputWanted;
}
@Override
public void run() {
TaskMonitor interactiveTaskMonitor = plugin.getInteractiveTaskMonitor();
JythonScript interactiveScript = plugin.getInteractiveScript();
Program program = plugin.getCurrentProgram();
// Setup transaction for the execution.
try (Transaction tx = program != null ? program.openTransaction("Jython command") : null) {
// Setup Ghidra state to be passed into interpreter
interactiveTaskMonitor.clearCancelled();
interactiveScript.setSourceFile(new ResourceFile(new File("jython")));
PluginTool tool = plugin.getTool();
interactiveScript.set(
new GhidraState(tool, tool.getProject(), program, plugin.getProgramLocation(),
plugin.getProgramSelection(), plugin.getProgramHighlight()),
interactiveTaskMonitor, new PrintWriter(plugin.getConsole().getStdOut()));
// Execute the command
moreInputWanted.set(false);
moreInputWanted.set(plugin.getInterpreter().push(cmd, plugin.getInteractiveScript()));
}
catch (PyException pye) {
String exceptionName = PyException.exceptionClassName(pye.type);
if (exceptionName.equalsIgnoreCase("exceptions.SystemExit")) {
plugin.reset();
}
else {
plugin.getConsole()
.getErrWriter()
.println(
"Suppressing exception: " + PyException.exceptionClassName(pye.type));
}
}
catch (StackOverflowError soe) {
plugin.getConsole().getErrWriter().println("Stack overflow!");
}
finally {
interactiveScript.end(false); // end any transactions the script may have started
}
}
}

105
Ghidra/Features/Jython/src/main/java/ghidra/jython/JythonPluginInputThread.java Normal file
View file

@ -0,0 +1,105 @@
/* ###
* IP: GHIDRA
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package ghidra.jython;
import java.io.*;
import java.util.concurrent.atomic.AtomicBoolean;
import ghidra.util.Msg;
/**
* Thread responsible for getting interactive lines of jython from the plugin.
* This class also kicks off the execution of that line in a new {@link JythonPluginExecutionThread}.
*/
class JythonPluginInputThread extends Thread {
private static int generationCount = 0;
private final JythonPlugin plugin;
private final AtomicBoolean moreInputWanted = new AtomicBoolean(false);
private final AtomicBoolean shutdownRequested = new AtomicBoolean(false);
private final InputStream consoleStdin;
private JythonPluginExecutionThread jythonExecutionThread;
/**
* Creates a new jython input thread that gets a line of jython input from the given plugin.
*
* @param plugin The jython plugin to get input from.
*/
JythonPluginInputThread(JythonPlugin plugin) {
super("Jython plugin input thread (generation " + ++generationCount + ")");
this.plugin = plugin;
this.consoleStdin = plugin.getConsole().getStdin();
}
/**
* Gets the last jython plugin execution thread that ran.
*
* @return The last jython plugin execution thread that ran. Could be null if one never ran.
*/
JythonPluginExecutionThread getJythonPluginExecutionThread() {
return jythonExecutionThread;
}
@Override
public void run() {
try (BufferedReader reader = new BufferedReader(new InputStreamReader(consoleStdin))) {
String line;
while (!shutdownRequested.get() && (line = reader.readLine()) != null) {
// Execute the line in a new thread
jythonExecutionThread =
new JythonPluginExecutionThread(plugin, line, moreInputWanted);
jythonExecutionThread.start();
try {
// Wait for the execution to finish
jythonExecutionThread.join();
}
catch (InterruptedException ie) {
// Hey we're back... a little earlier than expected, but there must be a reason.
// So we'll go quietly.
}
// Set the prompt appropriately
plugin.getConsole()
.setPrompt(
moreInputWanted.get() ? plugin.getInterpreter().getSecondaryPrompt()
: plugin.getInterpreter().getPrimaryPrompt());
}
}
catch (IOException e) {
Msg.error(JythonPluginInputThread.class,
"Internal error reading commands from interpreter console. Please reset the interpreter.",
e);
}
}
/**
* Causes the the background thread's run() loop to exit.
* <p>
* Causes background thread's exit by closing the inputstream it is looping on.
*/
void shutdown() {
try {
shutdownRequested.set(true);
consoleStdin.close();
}
catch (IOException e) {
// shouldn't happen, ignore
}
}
}

65
Ghidra/Features/Jython/src/main/java/ghidra/jython/JythonRun.java Normal file
View file

@ -0,0 +1,65 @@
/* ###
* IP: GHIDRA
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package ghidra.jython;
import java.io.IOException;
import org.python.util.jython;
import ghidra.GhidraApplicationLayout;
import ghidra.GhidraLaunchable;
import ghidra.framework.*;
import ghidra.util.Msg;
import ghidra.util.exception.CancelledException;
/**
* Launcher entry point for running Ghidra from within Jython.
*/
public class JythonRun implements GhidraLaunchable {
@Override
public void launch(GhidraApplicationLayout layout, String[] args) {
// Initialize the application
ApplicationConfiguration configuration = new HeadlessGhidraApplicationConfiguration();
Application.initializeApplication(layout, configuration);
// Setup jython home directory
try {
JythonUtils.setupJythonHomeDir();
}
catch (IOException e) {
Msg.showError(JythonRun.class, null, "Jython home directory", e.getMessage());
System.exit(1);
}
// Setup jython cache directory
try {
JythonUtils.setupJythonCacheDir(configuration.getTaskMonitor());
}
catch (IOException e) {
Msg.showError(JythonRun.class, null, "Jython cache directory", e.getMessage());
System.exit(1);
}
catch (CancelledException e) {
Msg.showError(JythonRun.class, null, "Operation cancelled", e.getMessage());
System.exit(1);
}
// Pass control to Jython
jython.main(args);
}
}

180
Ghidra/Features/Jython/src/main/java/ghidra/jython/JythonScript.java Normal file
View file

@ -0,0 +1,180 @@
/* ###
* IP: GHIDRA
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package ghidra.jython;
import java.io.PrintWriter;
import java.util.concurrent.atomic.AtomicBoolean;
import generic.jar.ResourceFile;
import ghidra.app.script.*;
import ghidra.app.services.ConsoleService;
import ghidra.framework.plugintool.PluginTool;
import ghidra.util.exception.AssertException;
/**
* A Jython version of a {@link GhidraScript}.
*/
public class JythonScript extends GhidraScript {
static final String JYTHON_INTERPRETER = "ghidra.jython.interpreter";
private AtomicBoolean interpreterRunning = new AtomicBoolean();
@Override
public void run() {
// Try to get the interpreter from an existing script state.
GhidraJythonInterpreter interpreter =
(GhidraJythonInterpreter) state.getEnvironmentVar(JYTHON_INTERPRETER);
// Are we being called from an already running JythonScript with existing state?
if (interpreter != null) {
runInExistingEnvironment(interpreter);
}
else {
runInNewEnvironment();
}
}
@Override
public void runScript(String scriptName, GhidraState scriptState) throws Exception {
GhidraJythonInterpreter interpreter =
(GhidraJythonInterpreter) state.getEnvironmentVar(JYTHON_INTERPRETER);
if (interpreter == null) {
interpreter = GhidraJythonInterpreter.get();
if (interpreter == null) {
throw new AssertException("Could not get Ghidra Jython interpreter!");
}
}
ResourceFile scriptSource = GhidraScriptUtil.findScriptByName(scriptName);
if (scriptSource != null) {
GhidraScriptProvider provider = GhidraScriptUtil.getProvider(scriptSource);
GhidraScript ghidraScript = provider.getScriptInstance(scriptSource, writer);
if (ghidraScript == null) {
throw new IllegalArgumentException("Script does not exist: " + scriptName);
}
if (scriptState == state) {
updateStateFromVariables();
}
if (ghidraScript instanceof JythonScript) {
ghidraScript.set(scriptState, monitor, writer);
JythonScript jythonScript = (JythonScript) ghidraScript;
interpreter.execFile(jythonScript.getSourceFile(), jythonScript);
}
else {
ghidraScript.execute(scriptState, monitor, writer);
}
if (scriptState == state) {
loadVariablesFromState();
}
return;
}
throw new IllegalArgumentException("Script does not exist: " + scriptName);
}
/**
* Runs this script in an existing interpreter environment.
*
* @param interpreter The existing interpreter to execute from.
*/
private void runInExistingEnvironment(GhidraJythonInterpreter interpreter) {
interpreter.execFile(sourceFile, this);
}
/**
* Runs this script in a new interpreter environment and sticks the new interpreter
* in the script state so it can be retrieved by scripts called from this script.
*/
private void runInNewEnvironment() {
// Create new interpreter and stick it in the script's state.
final GhidraJythonInterpreter interpreter = GhidraJythonInterpreter.get();
final PrintWriter stdout = getStdOut();
final PrintWriter stderr = getStdErr();
interpreter.setOut(stdout);
interpreter.setErr(stderr);
// We stick the interpreter in the state so that if the script calls runScript, that
// script will use the same interpreter. It is questionable whether or not we should do
// this (the new script will get all of the old script's variables), but changing it now
// could break people's scripts if they expect this behavior.
state.addEnvironmentVar(JYTHON_INTERPRETER, interpreter);
// Execute the script in a new thread.
JythonScriptExecutionThread executionThread =
new JythonScriptExecutionThread(this, interpreter, interpreterRunning);
interpreterRunning.set(true);
executionThread.start();
// Wait for the script be finish running
while (interpreterRunning.get() && !monitor.isCancelled()) {
Thread.yield();
sleep100millis();
}
if (interpreterRunning.get()) {
// We've been canceled. Interrupt the interpreter.
interpreter.interrupt(executionThread);
}
// Script is done. Make sure the output displays.
stderr.flush();
stdout.flush();
// Cleanup the interpreter, and remove it from the state (once it's cleaned it cannot be
// reused)
interpreter.cleanup();
state.removeEnvironmentVar(JYTHON_INTERPRETER);
}
private PrintWriter getStdOut() {
PluginTool tool = state.getTool();
if (tool != null) {
ConsoleService console = tool.getService(ConsoleService.class);
if (console != null) {
return console.getStdOut();
}
}
return new PrintWriter(System.out, true);
}
private PrintWriter getStdErr() {
PluginTool tool = state.getTool();
if (tool != null) {
ConsoleService console = tool.getService(ConsoleService.class);
if (console != null) {
return console.getStdErr();
}
}
return new PrintWriter(System.err, true);
}
private void sleep100millis() {
try {
Thread.sleep(100);
}
catch (InterruptedException e) {
// Don't care; will probably be called again
}
}
@Override
public String getCategory() {
return "Jython";
}
}

72
Ghidra/Features/Jython/src/main/java/ghidra/jython/JythonScriptExecutionThread.java Normal file
View file

@ -0,0 +1,72 @@
/* ###
* IP: GHIDRA
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package ghidra.jython;
import java.util.concurrent.atomic.AtomicBoolean;
import org.python.core.PyException;
/**
* Thread responsible for executing a jython script from a file.
*/
class JythonScriptExecutionThread extends Thread {
private JythonScript script;
private GhidraJythonInterpreter interpreter;
private AtomicBoolean interpreterRunning;
/**
* Creates a new jython script execution thread that executes the given jython script.
*
* @param script The jython script to execute.
* @param interpreter The jython interpreter to use for execution.
* @param interpreterRunning Gets set to indicate whether or not the interpreter is still running the script.
*/
JythonScriptExecutionThread(JythonScript script, GhidraJythonInterpreter interpreter,
AtomicBoolean interpreterRunning) {
super("Jython script execution thread");
this.script = script;
this.interpreter = interpreter;
this.interpreterRunning = interpreterRunning;
}
@Override
public void run() {
try {
interpreter.execFile(script.getSourceFile(), script);
}
catch (PyException pye) {
if (PyException.exceptionClassName(pye.type).equalsIgnoreCase(
"exceptions.SystemExit")) {
interpreter.printErr("SystemExit");
}
else {
pye.printStackTrace(); // this prints to the interpreter error stream.
}
}
catch (StackOverflowError soe) {
interpreter.printErr("Stack overflow!");
}
catch (IllegalStateException e) {
interpreter.printErr(e.getMessage());
}
finally {
interpreterRunning.set(false);
}
}
}

108
Ghidra/Features/Jython/src/main/java/ghidra/jython/JythonScriptProvider.java Normal file
View file

@ -0,0 +1,108 @@
/* ###
* IP: GHIDRA
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package ghidra.jython;
import java.io.*;
import java.util.regex.Pattern;
import generic.jar.ResourceFile;
import ghidra.app.script.*;
public class JythonScriptProvider extends GhidraScriptProvider {
private static final Pattern BLOCK_COMMENT = Pattern.compile("'''");
@Override
public void createNewScript(ResourceFile newScript, String category) throws IOException {
PrintWriter writer = new PrintWriter(new FileWriter(newScript.getFile(false)));
writeHeader(writer, category);
writer.println("");
writeBody(writer);
writer.println("");
writer.close();
}
/**
* {@inheritDoc}
*
* <p>
* In Jython this is a triple single quote sequence, "'''".
*
* @return the Pattern for Jython block comment openings
*/
@Override
public Pattern getBlockCommentStart() {
return BLOCK_COMMENT;
}
/**
* {@inheritDoc}
*
* <p>
* In Jython this is a triple single quote sequence, "'''".
*
* @return the Pattern for Jython block comment openings
*/
@Override
public Pattern getBlockCommentEnd() {
return BLOCK_COMMENT;
}
@Override
public String getCommentCharacter() {
return "#";
}
@Override
protected String getCertifyHeaderStart() {
return "## ###";
}
@Override
protected String getCertificationBodyPrefix() {
return "#";
}
@Override
protected String getCertifyHeaderEnd() {
return "##";
}
@Override
public String getDescription() {
return "Jython";
}
@Override
public String getExtension() {
return ".py";
}
@Override
public GhidraScript getScriptInstance(ResourceFile sourceFile, PrintWriter writer)
throws GhidraScriptLoadException {
try {
Class<?> clazz = Class.forName(JythonScript.class.getName());
GhidraScript script = (GhidraScript) clazz.getConstructor().newInstance();
script.setSourceFile(sourceFile);
return script;
}
catch (Exception e) {
throw new GhidraScriptLoadException(e);
}
}
}

108
Ghidra/Features/Jython/src/main/java/ghidra/jython/JythonUtils.java Normal file
View file

@ -0,0 +1,108 @@
/* ###
* IP: GHIDRA
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package ghidra.jython;
import java.io.*;
import ghidra.framework.Application;
import ghidra.util.exception.CancelledException;
import ghidra.util.task.TaskMonitor;
import utilities.util.FileUtilities;
/**
* Python utility method class.
*/
public class JythonUtils {
public static final String JYTHON_NAME = "jython-2.7.3";
public static final String JYTHON_CACHEDIR = "jython_cachedir";
public static final String JYTHON_SRC = "jython-src";
/**
* Sets up the jython home directory. This is the directory that has the "Lib" directory in it.
*
* @return The jython home directory.
* @throws IOException If there was a disk-related problem setting up the home directory.
*/
public static File setupJythonHomeDir() throws IOException {
File jythonModuleDir = Application.getMyModuleRootDirectory().getFile(false);
File jythonHomeDir =
Application.getModuleDataSubDirectory(jythonModuleDir.getName(), JYTHON_NAME)
.getFile(false);
if (!jythonHomeDir.exists()) {
throw new IOException("Failed to find the jython home directory at: " + jythonHomeDir);
}
System.setProperty("jython.home", jythonHomeDir.getAbsolutePath());
return jythonHomeDir;
}
/**
* Sets up the jython cache directory. This is a temporary space that jython source files
* get compiled to and cached. It should NOT be in the Ghidra installation directory, because
* some installations will not have the appropriate directory permissions to create new files in.
*
* @param monitor A monitor to use during the cache directory setup.
* @return The jython cache directory.
* @throws IOException If there was a disk-related problem setting up the cache directory.
* @throws CancelledException If the user cancelled the setup.
*/
public static File setupJythonCacheDir(TaskMonitor monitor)
throws CancelledException, IOException {
File devDir = new File(Application.getUserSettingsDirectory(), "dev");
File cacheDir = new File(devDir, JYTHON_CACHEDIR);
if (!FileUtilities.mkdirs(cacheDir)) {
throw new IOException("Failed to create the jython cache directory at: " + cacheDir);
}
File jythonSrcDestDir = new File(cacheDir, JYTHON_SRC);
if (!FileUtilities.createDir(jythonSrcDestDir)) {
throw new IOException(
"Failed to create the " + JYTHON_SRC + " directory at: " + jythonSrcDestDir);
}
File jythonModuleDir = Application.getMyModuleRootDirectory().getFile(false);
File jythonSrcDir = new File(jythonModuleDir, JYTHON_SRC);
if (!jythonSrcDir.exists()) {
try {
jythonSrcDir = Application.getModuleDataSubDirectory(jythonModuleDir.getName(),
JYTHON_SRC).getFile(false);
}
catch (FileNotFoundException e) {
throw new IOException("Failed to find the module's " + JYTHON_SRC + " directory");
}
}
try {
FileUtilities.copyDir(jythonSrcDir, jythonSrcDestDir, f -> f.getName().endsWith(".py"),
monitor);
}
catch (IOException e) {
throw new IOException(
"Failed to copy " + JYTHON_SRC + " files to: " + jythonSrcDestDir);
}
System.setProperty("python.cachedir.skip", "false");
System.setProperty("python.cachedir", cacheDir.getAbsolutePath());
System.setProperty("python.path", jythonSrcDestDir.getAbsolutePath());
return cacheDir;
}
}

41
Ghidra/Features/Jython/src/main/java/ghidra/jython/PyDevUtils.java Normal file
View file

@ -0,0 +1,41 @@
/* ###
* IP: GHIDRA
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package ghidra.jython;
import java.io.File;
import org.apache.commons.lang3.StringUtils;
public class PyDevUtils {
public static final int PYDEV_REMOTE_DEBUGGER_PORT = 5678;
/**
* Gets The PyDev source directory.
*
* @return The PyDev source directory, or null if it not known.
*/
public static File getPyDevSrcDir() {
String property = System.getProperty("eclipse.pysrc.dir");
return StringUtils.isNotBlank(property) ? new File(property) : null;
}
/**
* Prevent instantiation of utility class.
*/
private PyDevUtils() {
}
}

BIN
Ghidra/Features/Jython/src/main/resources/images/python.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 72 KiB

173
Ghidra/Features/Jython/src/test.slow/java/ghidra/jython/JythonCodeCompletionTest.java Normal file
View file

@ -0,0 +1,173 @@
/* ###
* IP: GHIDRA
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package ghidra.jython;
import static org.junit.Assert.*;
import java.io.File;
import java.io.IOException;
import java.nio.charset.Charset;
import java.util.*;
import java.util.stream.Collectors;
import org.apache.commons.io.FileUtils;
import org.junit.*;
import org.junit.rules.TemporaryFolder;
import generic.jar.ResourceFile;
import ghidra.app.plugin.core.console.CodeCompletion;
import ghidra.app.plugin.core.osgi.BundleHost;
import ghidra.app.script.GhidraScriptUtil;
import ghidra.test.AbstractGhidraHeadedIntegrationTest;
/**
* Tests for the Ghidra Python Interpreter's code completion functionality.
*/
public class JythonCodeCompletionTest extends AbstractGhidraHeadedIntegrationTest {
private String simpleTestProgram = """
my_int = 32
my_bool = True
my_string = 'this is a string'
my_list = ["a", 2, 5.3, my_string]
my_tuple = (1, 2, 3)
my_dictionary = {"key1": "1", "key2": 2, "key3": my_list}
mY_None = None
i = 5
def factorial(n):
return 1 if n == 0 else n * factorial(n-1)
def error_function():
raise IOError("An IO error occurred!")
class Employee:
def __init__(self, id, name):
self.id = id
self.name = name
def getId(self):
return self.id
def getName(self):
return self.name
employee = Employee(42, "Bob")
""".stripIndent();
@Rule
public TemporaryFolder tempScriptFolder = new TemporaryFolder();
private GhidraJythonInterpreter interpreter;
@Before
public void setUp() throws Exception {
GhidraScriptUtil.initialize(new BundleHost(), null);
interpreter = GhidraJythonInterpreter.get();
executeJythonProgram(simpleTestProgram);
}
@After
public void tearDown() throws Exception {
interpreter.cleanup();
GhidraScriptUtil.dispose();
}
@Test
public void testBasicCodeCompletion() {
// test the "insertion" field
// it should be equal to the full name of a variable we want to complete
List<String> completions = List.of("my_bool", "my_dictionary", "my_int", "my_list",
"mY_None", "my_string", "my_tuple");
assertCompletionsInclude("My", completions);
assertCompletionsInclude("employee.Get", List.of("getId", "getName"));
assertCompletionsInclude("('noise', (1 + fact", List.of("factorial"));
}
@Test
public void testCharsToRemoveField() {
// 'charsToRemove' field should be equal to the length of
// a part of variable/function/method name we are trying to complete here.
// This allows us to correctly put a completion in cases when we really
// just want to replace a piece of text (i.e. "CURRENTAddress" => "currentAddress")
// rather than simply 'complete' it.
assertCharsToRemoveEqualsTo("my_int", "my_int".length());
assertCharsToRemoveEqualsTo("employee.get", "get".length());
assertCharsToRemoveEqualsTo("('noise', (1 + fact", "fact".length());
assertCharsToRemoveEqualsTo("employee.", 0);
assertCharsToRemoveEqualsTo("employee.getId(", 0);
}
@Test
public void testCompletionsFromArbitraryCaretPositions() {
// '<caret>' designates the position of the caret
String testCase;
testCase = "MY_TUP<caret>";
assertCompletionsInclude(testCase, List.of("my_tuple"));
assertCharsToRemoveEqualsTo(testCase, "my_tup".length());
testCase = "employee.Get<caret>; (4, 2+2)";
assertCompletionsInclude(testCase, List.of("getId", "getName"));
assertCharsToRemoveEqualsTo(testCase, "Get".length());
testCase = "bar = e<caret> if 2+2==4 else 'baz'";
assertCompletionsInclude(testCase, List.of("error_function", "employee"));
assertCharsToRemoveEqualsTo(testCase, "e".length());
}
private void assertCompletionsInclude(String command, Collection<String> expectedCompletions) {
Set<String> completions = getCodeCompletions(command)
.stream()
.map(c -> c.getInsertion())
.collect(Collectors.toSet());
var missing = new HashSet<String>(expectedCompletions);
missing.removeAll(completions);
if (!missing.isEmpty()) {
Assert.fail("Could't find these completions: " + missing);
}
}
private void assertCharsToRemoveEqualsTo(String command, int expectedCharsToRemove) {
for (CodeCompletion comp : getCodeCompletions(command)) {
assertEquals(String.format("%s; field 'charsToRemove' ", comp), expectedCharsToRemove,
comp.getCharsToRemove());
}
}
private List<CodeCompletion> getCodeCompletions(String command) {
// extract the caret position and generate completions relative to that place
int caretPos = command.indexOf("<caret>");
command.replace("<caret>", "");
if (caretPos == -1) {
caretPos = command.length();
}
return interpreter.getCommandCompletions(command, false, caretPos);
}
private void executeJythonProgram(String code) {
try {
File tempFile = tempScriptFolder.newFile();
FileUtils.writeStringToFile(tempFile, code, Charset.defaultCharset());
interpreter.execFile(new ResourceFile(tempFile), null);
}
catch (IOException e) {
fail("couldn't create a test script: " + e.getMessage());
}
}
}

97
Ghidra/Features/Jython/src/test.slow/java/ghidra/jython/JythonInterpreterTest.java Normal file
View file

@ -0,0 +1,97 @@
/* ###
* IP: GHIDRA
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package ghidra.jython;
import static org.junit.Assert.*;
import java.io.ByteArrayOutputStream;
import org.junit.*;
import generic.jar.ResourceFile;
import ghidra.app.plugin.core.osgi.BundleHost;
import ghidra.app.script.GhidraScriptUtil;
import ghidra.test.AbstractGhidraHeadedIntegrationTest;
/**
* Tests the Ghidra python interpreter's functionality.
*/
public class JythonInterpreterTest extends AbstractGhidraHeadedIntegrationTest {
private ByteArrayOutputStream out;
private GhidraJythonInterpreter interpreter;
@Before
public void setUp() throws Exception {
out = new ByteArrayOutputStream();
GhidraScriptUtil.initialize(new BundleHost(), null);
interpreter = GhidraJythonInterpreter.get();
interpreter.setOut(out);
interpreter.setErr(out);
}
@After
public void tearDown() throws Exception {
out.reset();
interpreter.cleanup();
GhidraScriptUtil.dispose();
}
/**
* Tests that the interpreter's "push" method is working by executing a simple line of python.
*/
@Test
public void testJythonPush() {
final String str = "hi";
interpreter.push("print \"" + str + "\"", null);
assertEquals(out.toString().trim(), str);
}
/**
* Tests that the interpreter's "execFile" method is working by executing a simple file of python.
*/
@Test
public void testJythonExecFile() {
interpreter.execFile(new ResourceFile("ghidra_scripts/python_basics.py"), null);
assertTrue(out.toString().contains("Snoopy"));
}
/**
* Tests that our sitecustomize.py modules gets loaded by testing the custom help function
* that we install from there.
*/
@Test
public void testJythonSiteCustomize() {
interpreter.push("help", null);
assertTrue(out.toString().contains("Press 'F1'"));
}
/**
* Tests that cleaning the interpreter invalidates it.
*/
@Test
public void testJythonCleanupInvalidation() {
interpreter.cleanup();
try {
interpreter.push("pass", null);
fail("Push still worked after interpreter cleanup.");
}
catch (IllegalStateException e) {
// If everything worked, we should end up here.
}
}
}

62
Ghidra/Features/Jython/src/test.slow/java/ghidra/jython/JythonPluginTest.java Normal file
View file

@ -0,0 +1,62 @@
/* ###
* IP: GHIDRA
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package ghidra.jython;
import static org.junit.Assert.*;
import org.junit.*;
import ghidra.app.plugin.core.osgi.BundleHost;
import ghidra.app.script.GhidraScriptUtil;
import ghidra.framework.plugintool.PluginTool;
import ghidra.test.AbstractGhidraHeadedIntegrationTest;
import ghidra.test.TestEnv;
/**
* Tests the Python Plugin functionality.
*/
public class JythonPluginTest extends AbstractGhidraHeadedIntegrationTest {
private TestEnv env;
private PluginTool tool;
private JythonPlugin plugin;
@Before
public void setUp() throws Exception {
env = new TestEnv();
tool = env.getTool();
GhidraScriptUtil.initialize(new BundleHost(), null);
tool.addPlugin(JythonPlugin.class.getName());
plugin = env.getPlugin(JythonPlugin.class);
}
@After
public void tearDown() throws Exception {
GhidraScriptUtil.dispose();
env.dispose();
}
/**
* Tests that issuing a reset from the plugin resets the interpreter.
*/
@Test
public void testJythonPluginReset() {
GhidraJythonInterpreter origInterpreter = plugin.getInterpreter();
plugin.reset();
GhidraJythonInterpreter newInterpreter = plugin.getInterpreter();
assertNotSame(origInterpreter, newInterpreter);
}
}

237
Ghidra/Features/Jython/src/test.slow/java/ghidra/jython/JythonScriptInfoTest.java Normal file
View file

@ -0,0 +1,237 @@
/* ###
* IP: GHIDRA
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package ghidra.jython;
import static org.junit.Assert.*;
import java.io.*;
import java.nio.file.Files;
import java.nio.file.Path;
import javax.swing.KeyStroke;
import org.junit.*;
import generic.jar.ResourceFile;
import ghidra.app.plugin.core.osgi.BundleHost;
import ghidra.app.script.GhidraScriptUtil;
import ghidra.app.script.ScriptInfo;
import ghidra.test.AbstractGhidraHeadedIntegrationTest;
public class JythonScriptInfoTest extends AbstractGhidraHeadedIntegrationTest {
@Before
public void setUp() throws Exception {
GhidraScriptUtil.initialize(new BundleHost(), null);
Path userScriptDir = java.nio.file.Paths.get(GhidraScriptUtil.USER_SCRIPTS_DIR);
if (Files.notExists(userScriptDir)) {
Files.createDirectories(userScriptDir);
}
}
@After
public void tearDown() throws Exception {
GhidraScriptUtil.dispose();
}
@Test
public void testDetailedJythonScript() {
String descLine1 = "This script exists to check that the info on";
String descLine2 = "a script that has extensive documentation is";
String descLine3 = "properly parsed and represented.";
String author = "Fake Name";
String categoryTop = "Test";
String categoryBottom = "ScriptInfo";
String keybinding = "ctrl shift COMMA";
String menupath = "File.Run.Detailed Script";
String importPackage = "detailStuff";
ResourceFile scriptFile = null;
try {
//@formatter:off
scriptFile = createTempPyScriptFileWithLines(
"'''",
"This is a test block comment. It will be ignored.",
"@category NotTheRealCategory",
"'''",
"#" + descLine1,
"#" + descLine2,
"#" + descLine3,
"#@author " + author,
"#@category " + categoryTop + "." + categoryBottom,
"#@keybinding " + keybinding,
"#@menupath " + menupath,
"#@importpackage " + importPackage,
"print('for a blank class, it sure is well documented!')");
//@formatter:on
}
catch (IOException e) {
fail("couldn't create a test script: " + e.getMessage());
}
ScriptInfo info = GhidraScriptUtil.newScriptInfo(scriptFile);
String expectedDescription = descLine1 + " \n" + descLine2 + " \n" + descLine3 + " \n";
assertEquals(expectedDescription, info.getDescription());
assertEquals(author, info.getAuthor());
assertEquals(KeyStroke.getKeyStroke(keybinding), info.getKeyBinding());
assertEquals(menupath.replace(".", "->"), info.getMenuPathAsString());
assertEquals(importPackage, info.getImportPackage());
String[] actualCategory = info.getCategory();
assertEquals(2, actualCategory.length);
assertEquals(categoryTop, actualCategory[0]);
assertEquals(categoryBottom, actualCategory[1]);
}
@Test
public void testJythonScriptWithBlockComment() {
String description = "Script with a block comment at the top.";
String category = "Test";
ResourceFile scriptFile = null;
try {
//@formatter:off
scriptFile = createTempPyScriptFileWithLines(
"'''",
"This is a test block comment. It will be ignored.",
"@category NotTheRealCategory",
"'''",
"#" + description,
"#@category " + category,
"print 'hello!'");
//@formatter:on
}
catch (IOException e) {
fail("couldn't create a test script: " + e.getMessage());
}
ScriptInfo info = GhidraScriptUtil.newScriptInfo(scriptFile);
assertEquals(description + " \n", info.getDescription());
String[] actualCategory = info.getCategory();
assertEquals(1, actualCategory.length);
assertEquals(category, actualCategory[0]);
}
@Test
public void testJythonScriptWithBlockCommentAndCertifyHeader() {
String description = "Script with a block comment at the top.";
String category = "Test";
ResourceFile scriptFile = null;
try {
//@formatter:off
scriptFile = createTempPyScriptFileWithLines(
"## ###",
"# IP: GHIDRA",
"# ",
"# Some license text...",
"# you may not use this file except in compliance with the License.",
"# ",
"# blah blah blah",
"##",
"",
"'''",
"This is a test block comment. It will be ignored.",
"@category NotTheRealCategory",
"'''",
"#" + description,
"#@category " + category,
"print 'hello!'");
//@formatter:on
}
catch (IOException e) {
fail("couldn't create a test script: " + e.getMessage());
}
ScriptInfo info = GhidraScriptUtil.newScriptInfo(scriptFile);
assertEquals(description + " \n", info.getDescription());
String[] actualCategory = info.getCategory();
assertEquals(1, actualCategory.length);
assertEquals(category, actualCategory[0]);
}
@Test
public void testJythonScriptWithoutBlockComment() {
String description = "Script without a block comment at the top.";
String category = "Test";
ResourceFile scriptFile = null;
try {
//@formatter:off
scriptFile = createTempPyScriptFileWithLines(
"#" + description,
"#@category " + category,
"print 'hello!'");
//@formatter:on
}
catch (IOException e) {
fail("couldn't create a test script: " + e.getMessage());
}
ScriptInfo info = GhidraScriptUtil.newScriptInfo(scriptFile);
assertEquals(description + " \n", info.getDescription());
String[] actualCategory = info.getCategory();
assertEquals(1, actualCategory.length);
assertEquals(category, actualCategory[0]);
}
@Test
public void testJythonScriptWithSingleLineBlockComment() {
String description = "Script with a block comment at the top.";
String category = "Test";
ResourceFile scriptFile = null;
try {
//@formatter:off
scriptFile = createTempPyScriptFileWithLines(
"'''This is a test block comment. It will be ignored.'''",
"#" + description,
"#@category " + category,
"print 'hello!'");
//@formatter:on
}
catch (IOException e) {
fail("couldn't create a test script: " + e.getMessage());
}
ScriptInfo info = GhidraScriptUtil.newScriptInfo(scriptFile);
assertEquals(description + " \n", info.getDescription());
String[] actualCategory = info.getCategory();
assertEquals(1, actualCategory.length);
assertEquals(category, actualCategory[0]);
}
private ResourceFile createTempPyScriptFileWithLines(String... lines) throws IOException {
File scriptDir = new File(GhidraScriptUtil.USER_SCRIPTS_DIR);
File tempFile = File.createTempFile(testName.getMethodName(), ".py", scriptDir);
tempFile.deleteOnExit();
ResourceFile tempResourceFile = new ResourceFile(tempFile);
PrintWriter writer = new PrintWriter(tempResourceFile.getOutputStream());
for (String line : lines) {
writer.println(line);
}
writer.close();
return tempResourceFile;
}
}

145
Ghidra/Features/Jython/src/test.slow/java/ghidra/jython/JythonScriptTest.java Normal file
View file

@ -0,0 +1,145 @@
/* ###
* IP: GHIDRA
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package ghidra.jython;
import static org.junit.Assert.*;
import java.io.*;
import java.util.concurrent.atomic.AtomicReference;
import org.junit.*;
import generic.jar.ResourceFile;
import ghidra.app.plugin.core.console.ConsolePlugin;
import ghidra.app.plugin.core.osgi.BundleHost;
import ghidra.app.script.GhidraScriptUtil;
import ghidra.app.script.GhidraState;
import ghidra.app.services.ConsoleService;
import ghidra.framework.Application;
import ghidra.framework.plugintool.PluginTool;
import ghidra.test.AbstractGhidraHeadedIntegrationTest;
import ghidra.test.TestEnv;
import ghidra.util.task.TaskMonitor;
/**
* Tests the Python script functionality.
*/
public class JythonScriptTest extends AbstractGhidraHeadedIntegrationTest {
private TestEnv env;
private PluginTool tool;
private ConsoleService console;
@Before
public void setUp() throws Exception {
env = new TestEnv();
tool = env.getTool();
GhidraScriptUtil.initialize(new BundleHost(), null);
tool.addPlugin(ConsolePlugin.class.getName());
console = tool.getService(ConsoleService.class);
}
@After
public void tearDown() throws Exception {
GhidraScriptUtil.dispose();
env.dispose();
}
/**
* Tests that Jython scripts are running correctly.
*
* @throws Exception If an exception occurred while trying to run the script.
*/
@Test
public void testJythonScript() throws Exception {
String script = "ghidra_scripts/python_basics.py";
try {
String output = runPythonScript(Application.getModuleFile("Jython", script));
assertTrue(output.contains("Snoopy"));
}
catch (FileNotFoundException e) {
fail("Could not find jython script: " + script);
}
catch (Exception e) {
fail("Exception occurred trying to run script: " + e.getMessage());
}
}
/**
* Tests that Jython scripts are running correctly.
*
* @throws Exception If an exception occurred while trying to run the script.
*/
@Test
public void testJythonInterpreterGoneFromState() throws Exception {
String script = "ghidra_scripts/python_basics.py";
try {
GhidraState state =
new GhidraState(env.getTool(), env.getProject(), null, null, null, null);
runPythonScript(Application.getModuleFile("Jython", script), state);
assertTrue(state.getEnvironmentVar(JythonScript.JYTHON_INTERPRETER) == null);
}
catch (FileNotFoundException e) {
fail("Could not find python script: " + script);
}
catch (Exception e) {
fail("Exception occurred trying to run script: " + e.getMessage());
}
}
/**
* Runs the given Python script.
*
* @param scriptFile The Python script to run.
* @return The console output of the script.
* @throws Exception If an exception occurred while trying to run the script.
*/
private String runPythonScript(ResourceFile scriptFile) throws Exception {
GhidraState state =
new GhidraState(env.getTool(), env.getProject(), null, null, null, null);
return runPythonScript(scriptFile, state);
}
/**
* Runs the given Python script with the given initial state.
*
* @param scriptFile The Python script to run.
* @param state The initial state of the script.
* @return The console output of the script.
* @throws Exception If an exception occurred while trying to run the script.
*/
private String runPythonScript(ResourceFile scriptFile, GhidraState state) throws Exception {
runSwing(() -> console.clearMessages());
JythonScriptProvider scriptProvider = new JythonScriptProvider();
PrintWriter writer = new PrintWriter(new ByteArrayOutputStream());
JythonScript script = (JythonScript) scriptProvider.getScriptInstance(scriptFile, writer);
script.set(state, TaskMonitor.DUMMY, writer);
script.run();
waitForSwing();
AtomicReference<String> ref = new AtomicReference<>();
runSwing(() -> {
String text = console.getText(0, console.getTextLength());
ref.set(text);
});
String text = ref.get();
return text;
}
}