Skip to main content
aboutsummaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorJeffrey Overbey2008-08-25 17:11:38 +0000
committerJeffrey Overbey2008-08-25 17:11:38 +0000
commit6122861c793f99e6cc38f02e5af3ae7282d6f6d8 (patch)
tree23b0ff693b924a6597a9df5a0ee3676efe8ab06d
parent592b61d1d319461cd242ef08ee0f3b1d5284728b (diff)
downloadorg.eclipse.photran-6122861c793f99e6cc38f02e5af3ae7282d6f6d8.tar.gz
org.eclipse.photran-6122861c793f99e6cc38f02e5af3ae7282d6f6d8.tar.xz
org.eclipse.photran-6122861c793f99e6cc38f02e5af3ae7282d6f6d8.zip
Committed Nick's revisions
-rw-r--r--org.eclipse.photran-dev-docs/dev-guide/app-cvs.ltx-inc87
-rw-r--r--org.eclipse.photran-dev-docs/dev-guide/app-error-parsers.ltx-inc74
-rw-r--r--org.eclipse.photran-dev-docs/dev-guide/app-obfuscate-fortran.ltx-inc179
-rw-r--r--org.eclipse.photran-dev-docs/dev-guide/app-rename-tests.ltx-inc185
-rw-r--r--org.eclipse.photran-dev-docs/dev-guide/authors.ltx1
-rw-r--r--org.eclipse.photran-dev-docs/dev-guide/cdt.ltx-inc175
-rw-r--r--org.eclipse.photran-dev-docs/dev-guide/cvs-instructions.ltx9
-rw-r--r--org.eclipse.photran-dev-docs/dev-guide/cvs-instructions.pdfbin62896 -> 69549 bytes
-rw-r--r--org.eclipse.photran-dev-docs/dev-guide/dev-guide.ltx54
-rw-r--r--org.eclipse.photran-dev-docs/dev-guide/editor.ltx-inc154
-rw-r--r--org.eclipse.photran-dev-docs/dev-guide/images/diagrams/editor_hierarchy.grafflebin0 -> 2188 bytes
-rw-r--r--org.eclipse.photran-dev-docs/dev-guide/images/diagrams/parser_chain.grafflebin0 -> 2671 bytes
-rw-r--r--org.eclipse.photran-dev-docs/dev-guide/images/editor_hierarchy.pngbin0 -> 98251 bytes
-rw-r--r--org.eclipse.photran-dev-docs/dev-guide/images/parser_chain.pngbin0 -> 211722 bytes
-rw-r--r--org.eclipse.photran-dev-docs/dev-guide/images/simple_fortran_AST.pngbin0 -> 43161 bytes
-rw-r--r--org.eclipse.photran-dev-docs/dev-guide/intro.ltx-inc35
-rw-r--r--org.eclipse.photran-dev-docs/dev-guide/parsing.ltx-inc717
-rw-r--r--org.eclipse.photran-dev-docs/dev-guide/plugins.ltx-inc218
-rw-r--r--org.eclipse.photran-dev-docs/dev-guide/refactoring.ltx-inc380
19 files changed, 1740 insertions, 528 deletions
diff --git a/org.eclipse.photran-dev-docs/dev-guide/app-cvs.ltx-inc b/org.eclipse.photran-dev-docs/dev-guide/app-cvs.ltx-inc
index 3f072551..5847e6c2 100644
--- a/org.eclipse.photran-dev-docs/dev-guide/app-cvs.ltx-inc
+++ b/org.eclipse.photran-dev-docs/dev-guide/app-cvs.ltx-inc
@@ -1,10 +1,11 @@
% Getting the Photran Sources from CVS
-\textit{Last Updated 07/21/08}
+\emph{BEFORE YOU BEGIN: Make sure you are running \textbf{Eclipse 3.4} and a
+\textbf{Java 5} or later JVM. We recommend the
+\href{http://www.eclipse.org/downloads/packages/eclipse-rcpplug-developers/ganymeder}{Eclipse for RCP/Plug-in Developers Package}.}
-\emph{BEFORE YOU BEGIN: Make sure you are running \textbf{Eclipse 3.4} and a \textbf{Java 5} or later JVM.}
-
-If you already have CDT 5.0 installed and do not need to edit the CDT source code, Part~I can be skipped.
+If you already have CDT 5.0 installed and do not need to edit the CDT source
+code, Part~I can be skipped.
\newcounter{running_counter}
\textbf{Part~I. Check out the CDT 5.0 sources from CVS}
@@ -20,17 +21,27 @@ If you already have CDT 5.0 installed and do not need to edit the CDT source cod
Password: & (no password) \\
Connection type: & pserver \\
\end{tabular}
- \item Right-click on :pserver:anonymous@dev.eclipse.org:/cvsroot/tools, and choose
- Refresh Branches...
- \item In the dialog box, scroll down, check the box next to org.eclipse.cdt, and click Finish.
- When prompted, click on Search Deeply. You will have to wait for a few minutes (possibly up to 10 minutes) for processing to complete and the dialog to disappear.
- \item Now, in the CVS Repositories view
+ \item In the CVS Repositories view
\begin{itemize}
\item Expand ``:pserver:anonymous@dev.eclipse.org:/cvsroot/tools''
- \item Then expand ``Versions''
- \item Then expand ``org.eclipse.cdt CDT\_5\_0\_0''
- \item Then expand ``all''
+ \item Then expand ``HEAD''
\end{itemize}
+ \item Right-click on ``org.eclipse.cdt''
+ \item Select ``Configure Branches and Versions\ldots''
+ \item Under ``Browse files for tags'', expand ``all'', then expand
+ ``org.eclipse.cdt'', then click on the .project file
+ \item Under ``New tags found in the selected files'', click on the Deselect
+ All button, then check cdt\_5\_0 in the list above it
+ \item Click Add Checked Tags
+ \item Click OK
+ \item Now, in the CVS Repositories view
+ \begin{itemize}
+ \item Expand ``:pserver:anonymous@dev.eclipse.org:/cvsroot/tools''
+ \item Then expand ``Branches''
+ \item Then expand ``cdt\_5\_0''
+ \item Then expand ``org.eclipse.cdt cdt\_5\_0\_0''
+ \item Then expand ``all''
+ \end{itemize}
\item Click on the first entry under ``all''
(it should be org.eclipse.cdt), then shift-click on the last entry
under ``all'' (it should be org.eclipse.cdt.ui.tests).
@@ -49,6 +60,7 @@ If you already have CDT 5.0 installed and do not need to edit the CDT source cod
\setcounter{enumi}{\value{running_counter}}
\item In Eclipse, switch to the CVS Repository Exploring perspective.
\item Right-click the CVS Repositories view; choose New, Repository Location
+ \pagebreak
\item Enter the following information, then click Finish: \\
\textit{If you are a Photran committer:} \\
\begin{tabular}{ll}
@@ -68,7 +80,7 @@ If you already have CDT 5.0 installed and do not need to edit the CDT source cod
org.eclipse.photran
\item Check out the following projects under org.eclipse.photran:
\begin{itemize}
- \item org.eclipse.photran-dev-docs
+ \item org.eclipse.photran-dev-docs (if you intend to contribute to the documentation)
\item org.eclipse.photran.cdtinterface
\item org.eclipse.photran.core
\item org.eclipse.photran.core.intel
@@ -76,7 +88,6 @@ If you already have CDT 5.0 installed and do not need to edit the CDT source cod
\item org.eclipse.photran.core.vpg.tests
\item org.eclipse.photran.core.vpg.tests.failing
\item org.eclipse.photran.errorparsers.xlf
- \item org.eclipse.photran.intel-feature
\item org.eclipse.photran.managedbuilder.core
\item org.eclipse.photran.managedbuilder.gnu.ui
\item org.eclipse.photran.managedbuilder.intel.ui
@@ -84,16 +95,15 @@ If you already have CDT 5.0 installed and do not need to edit the CDT source cod
\item org.eclipse.photran.managedbuilder.xlf.ui
\item org.eclipse.photran.ui
\item org.eclipse.photran.ui.vpg
- \item org.eclipse.photran.vpg-feature
\end{itemize}
- (The debug and launch plug-ins are not part of Photran 4.0 and will
- not compile. The analysis and refactoring plug-ins have been deprecated;
- they do not contain any files, since that functionality is in the
- VPG plug-ins.)
+ (The debug and launch plug-ins are not part of Photran 4.0 and will not
+ compile. The analysis and refactoring plug-ins have been deprecated;
+ they do not contain any files, since that functionality is in the VPG
+ plug-ins.)
- The sources should all compile (albeit with lots of warnings).
\setcounter{running_counter}{\value{enumi}}
\end{enumerate}
+The sources should all compile (albeit with lots of warnings).
\vspace{1em}
\noindent\textbf{Part~III. Running the test cases}
@@ -101,14 +111,41 @@ If you already have CDT 5.0 installed and do not need to edit the CDT source cod
\begin{enumerate}
\setcounter{enumi}{\value{running_counter}}
\item In Package Explorer view, select the \texttt{org.eclipse.photran.core.vpg.tests} project.
- \item Right-click on that project and select Run As $>$ Run Configurations... A dialog will appear.
+ \item Right-click on that project and select Run As $>$ Run Configurations\ldots. A dialog will appear.
\item In that dialog, create a new \textbf{JUnit Plug-in Test} launch configuration. Call it ``Photran-Tests''.
\item For the configuration that you have just created, switch to the ``Environment'' tab and create a new variable called ``TESTING'' with a value of 1.
\item Select ``Run'' to run the tests. To run the tests again, just launch the ``Photran-Tests'' configuration from the Eclipse Run menu.
\vspace{.5em}
- \noindent\textit{\textbf{Note.} Some JUnit tests for the parser and refactoring
- engine require closed-source code that is not available in CVS. A warning will
- appear in the JUnit runner if this code is not available.}
-
+ \noindent\textit{\textbf{Note.} Some JUnit tests for the parser and
+ refactoring engine require closed-source code that is not available in CVS.
+ A warning will appear in the JUnit runner if this code is not available.}
+ \setcounter{running_counter}{\value{enumi}}
\end{enumerate}
+
+\vspace{1em}
+\noindent\textbf{Part~IV. Deploying Photran Feature}
+\begin{enumerate}
+ \setcounter{enumi}{\value{running_counter}}
+ \item If you are interested in creating a \emph{deployable feature} for
+ Photran, you also need to check out these \textbf{additional} four
+ projects from CVS:
+ \begin{itemize}
+ \item org.eclipse.photran-feature
+ \item org.eclipse.photran.intel-feature
+ \item org.eclipse.photran.vpg-feature
+ \item org.eclipse.photran.xlf-feature
+ \end{itemize}
+ \item In Eclipse, select File $>$ Export\ldots
+ \item In the dialog that pops-up, select Plug-in Development $>$ Deployable features.
+ \item Click next.
+ \item In the list, select
+ \begin{itemize}
+ \item org.eclipse.photran\_feature (4.0.4)
+ \item org.eclipse.photran.intel (4.0.4)
+ \item org.eclipse.photran.vpg\_feature (4.0.4)
+ \item org.eclipse.photran.xlf\_feature (4.0.4)
+ \end{itemize}
+ \item Specify a destination folder to export those features. Click Finish.
+ \item The Photran features are ready for deployment.
+\end{enumerate} \ No newline at end of file
diff --git a/org.eclipse.photran-dev-docs/dev-guide/app-error-parsers.ltx-inc b/org.eclipse.photran-dev-docs/dev-guide/app-error-parsers.ltx-inc
index 2f13cd89..aa9c0a91 100644
--- a/org.eclipse.photran-dev-docs/dev-guide/app-error-parsers.ltx-inc
+++ b/org.eclipse.photran-dev-docs/dev-guide/app-error-parsers.ltx-inc
@@ -1,54 +1,56 @@
% Creating an Error Parser
-\textit{Last Updated 4/4/07}
-
-Error parsers scan the output of \texttt{make} for error messages
-for a particular compiler.
-When they see an error message they can recognize, they extract the
-filename, line number, and error message, and use it to populate the
+Error parsers scan the output of \texttt{make} for error messages for a
+particular compiler. When they see an error message they can recognize, they
+extract the filename, line number, and error message, and use it to populate the
Problems view.
For an example, see \texttt{IntelFortranErrorParser}.
-(It's a mere 74 lines.)
+(It's a mere 60 lines.)
+
+To create a new error parser, do the following.
-To create a new error parser, do the following.
-\begin{itemize}
+\begin{itemize}
+
\item We will assume that your error parser class will be in the
-\texttt{errorparsers} folder in the org.eclipse.photran.core
-plug-in and added to the
-\texttt{org.eclipse.photran.internal.errorparsers} package.
+\texttt{errorparsers} folder in the \texttt{org.eclipse.photran.core} plug-in
+and added to the \\\texttt{org.eclipse.photran.internal.errorparsers} package.
\item Define a class implementing \texttt{IErrorParser}
\item Implement \texttt{public boolean processLine(String line,
-ErrorParserManager eoParser)}
- which should always return false because ErrorParserManager appears not to
- use the result in a rational way
-
-\item In org.eclipse.photran.core's \texttt{plugin.xml}, find the place
-where we define all of the Fortran error parsers. Basically, copy an
-existing one. Your addition will look something like this:
-\begin{verbatim}
- <extension
- id="IntelFortranErrorParser"
- name="Photran Error Parser for Some New Fortran Compiler"
- point="org.eclipse.cdt.core.ErrorParser">
- <errorparser
- class="org.eclipse.photran.internal.errorparsers.MyErrorParser">
- </errorparser>
- </extension>
-\end{verbatim}
+ErrorParserManager eoParser)} which should always return false because
+ErrorParserManager appears not to use the result in a rational way
+
+\item In org.eclipse.photran.core's \texttt{plugin.xml}, find the place where we
+define all of the Fortran error parsers. Basically, copy an existing one. Your
+addition will look something like this:
+
+\pagebreak
+
+\begin{lstlisting}[frame=lines, language=xml]
+<extension
+ id="IntelFortranErrorParser"
+ name="Photran Error Parser for Some New Fortran Compiler"
+ point="org.eclipse.cdt.core.ErrorParser">
+ <errorparser
+ class="org.eclipse.photran.internal.errorparsers.MyErrorParser">
+ </errorparser>
+</extension>
+\end{lstlisting}
\item Your new error parser will appear in the error parser list in the
Preferences automatically, and it will be automatically added to new projects.
-For existing projects, you will need to open the project properties dialog
-and add the new error parser to the project manually.
+For existing projects, you will need to open the project properties dialog and
+add the new error parser to the project manually.
\end{itemize}
-\textbf{Note.} Error parsers to not have to be implemented in the Photran
-Core plug-in. In fact, they do not have to be implemented in Photran at all.
-If you create a brand new plug-in, you can specify org.eclipse.cdt.core
+\textbf{Note.} Error parsers do not have to be implemented in the Photran Core
+plug-in. In fact, they do not have to be implemented as part of Photran at all.
+If you create a brand new plug-in, you can specify \texttt{org.eclipse.cdt.core}
as a dependency, include the above XML snippet in your plug-in's
-\texttt{plugin.xml}, and include your custom error parser class in that
-plug-in.
+\texttt{plugin.xml}, and include your custom error parser class in that plug-in.
+The plug-in system for Eclipse will recognize your plug-in, detect that it
+extends the \texttt{org.eclipse.cdt.core.ErrorParser} extension point, and add
+it to the list of implemented error parsers automatically.
diff --git a/org.eclipse.photran-dev-docs/dev-guide/app-obfuscate-fortran.ltx-inc b/org.eclipse.photran-dev-docs/dev-guide/app-obfuscate-fortran.ltx-inc
new file mode 100644
index 00000000..0b712e89
--- /dev/null
+++ b/org.eclipse.photran-dev-docs/dev-guide/app-obfuscate-fortran.ltx-inc
@@ -0,0 +1,179 @@
+\section{Introduction}
+
+Contributing a new refactoring to Photran is best done by following a working
+example.
+
+This paragraph describes the general approach: First, an action must be added to
+both the editor popup menu \textbf{and} the Refactor menu in the menu bar by
+modifying the plugin.xml file. Then, the action delegate and its accompanying
+refactoring wizard have to be coded; these two classes are responsible for
+populating the user interface of the refactoring wizard dialog. Finally, the
+actual Fortran refactoring itself has to be coded.
+
+The remaining sections go into the details of each of those steps based on a
+simple (but not useful) refactoring example: obfuscating Fortran by removing the
+comments and adding redundant comments to the header. The source code is
+available from
+\url{http://subversion.cs.uiuc.edu/pub/svn/FruitFly/edu.uiuc.nchen.obfuscator/trunk/}
+
+\section{Modifying the plugin.xml}
+
+There are \textbf{four} extensions points (from the Eclipse core) that our plug-in needs to extend:
+
+\begin{description}
+
+ \item [\texttt{org.eclipse.ui.commands}] Creates a new command \emph{category}
+to represent our refactoring. This category will be referenced by the other
+extensions in the plugin.xml file.
+
+ \item [\texttt{org.eclipse.ui.actionSets}] This extension point is used to add
+menus and menu items to the Fortran perspective.
+
+ \item [\texttt{org.eclipse.ui.actionSetPartAssociations}] Allows our
+refactoring to be visible/enabled in the context of the Fortran editor.
+
+ \item [\texttt{org.eclipse.ui.popupMenus}] Displays our refactoring in the
+pop-up menu that appears during a right-click.
+
+ \item [\texttt{org.eclipse.ui.bindings}] (Optional) Allows our refactoring to
+be invoked via keyboard shortcuts. For instance the Fortran Rename Refactoring
+is bound to the Alt + Shift + R keyboard shortcut, which is the same as the one
+for the Java Rename Refactoring.
+
+\end{description}
+
+Please refer to the documentation and schema description for each extension point; the documentation is available from Help $>$ Help Contents in Eclipse.
+
+Fortran currently does \textbf{not} use the newer \texttt{org.eclipse.ui.menus}
+extension points (introduced in Eclipse 3.3) for adding menus, menu items and
+pop-up menus.
+
+It is possible to use the newer \texttt{org.eclipse.ui.menus} extension point if
+desired, but this chapter uses the older extension points to remain consistent
+with how Photran is doing it.
+
+For more information, see the plugin.xml file of our refactoring example.
+
+\section{Creating an Action Delegate and a Refactoring Wizard}
+
+The \texttt{org.eclipse.ui.actionSets} and \texttt{org.eclipse.ui.popupMenus}
+extension points that were extended in our plugin.xml file require a reference
+to action delegate class that we need to provide.
+
+For a Fortran refactoring, our action delegate should extend the
+\\\texttt{AbstractFortranRefactoringActionDelegate} class \textbf{and} implement
+the \\\texttt{IWorkbenchWindowActionDelegate} and \texttt{IEditorActionDelegate}
+interfaces.
+
+The most important method in our action delegate class is the
+\textbf{constructor}. The constructor has to be done in a particular way so that
+everything is setup correctly. Listing~\ref{lst:obfuscateaction} shows how the constructor needs to be setup.
+
+\begin{code}
+\begin{lstlisting}
+public class ObfuscateAction extends AbstractFortranRefactoringActionDelegate
+ implements IWorkbenchWindowActionDelegate, IEditorActionDelegate {
+
+ public ObfuscateAction() {
+ super(ObfuscateRefactoring.class, ObfuscateRefactoringWizard.class);
+ }
+
+ ...
+}
+\end{lstlisting}
+\caption{\texttt{ObfuscateAction} for our simple refactoring example}
+\label{lst:obfuscateaction}
+\end{code}
+
+Inside our constructor, we need to call the parent constructor that takes
+\textbf{two} parameters: the class of the actual refactoring object (e.g.
+ObfuscateRefactoring) and the class of the actual refactoring wizard (e.g.
+ObfuscateRefactoringWizard). The parent class will dynamically create the
+refactoring object and refactoring wizard using Java reflection.
+
+Our refactoring wizard needs to be a subclass of
+\texttt{AbstractFortranRefactoringWizard}. The only method that we are required
+to implement is the \texttt{doAddUserInputPages} method. This page is
+responsible for creating a page for the wizard. For instance, a refactoring such
+as rename refactoring requires the user to provide a new name. So the
+\texttt{doAddUserInputPages} is responsible for creating the interface for that.
+
+Ideally, if our refactoring does not require the user to provide any input, it
+should just have an empty \texttt{doAddUserInputPages} method. However, because
+of a bug in the Mac OS X version of Eclipse, it is necessary to add a
+\emph{dummy} page. Without this dummy page the refactoring will cause the entire
+Eclipse UI to lock up on Mac OS X. Listing~\ref{lst:dummy_wizard_page} shows how
+to add a dummy input page. \\
+
+\begin{code}
+\begin{lstlisting}
+protected void doAddUserInputPages() {
+ addPage(new UserInputWizardPage(refactoring.getName()) {
+
+ public void createControl(Composite parent) {
+ Composite top = new Composite(parent, SWT.NONE);
+ initializeDialogUnits(top);
+ setControl(top);
+
+ top.setLayout(new GridLayout(1, false));
+
+ Label lbl = new Label(top, SWT.NONE);
+ lbl.setText("Click OK to obfuscate the current Fortran file.
+ To see what changes will be made, click Preview.");
+
+ }
+ });
+}
+\end{lstlisting}
+\caption{Adding a \texttt{dummy} wizard input page}
+\label{lst:dummy_wizard_page}
+\end{code}
+
+\section{Creating the Actual Refactoring}
+
+Section~\ref{sec:structure_of_a_fortran_refactoring} gives a good overview of
+the \textbf{four} methods that a Fortran refactoring needs to implement. And
+Section~\ref{sec:refactoring_caveats} gives an overview of things to avoid while
+performing a refactoring. Our example refactoring conforms to the lessons in
+both those sections.
+
+Here we briefly describe the four methods in our example:
+
+\begin{description}
+
+ \item [getName] This just returns the text ``Obfuscate Fortran Code''
+describing our refactoring. This text will be used as the title of the
+refactoring wizard dialog.
+
+ \item [doCheckInitialConditions] Our simple refactoring does not have any
+\emph{real} initial conditions. Our refactoring can proceed as long as the
+current file can be parsed as valid Fortran source code. This is automatically
+checked by the \texttt{FortranRefactoring} parent class.
+
+ Instead we use this method as a hook to perform some simple program analysis --
+acquiring the names of all the functions and subroutines in the current file. We
+will print these names later as part of the header comment.
+
+ \item [doCheckFinalConditions] Since we do not require the user to provide any
+additional input, there are no final conditions to check.
+
+ \item [doCreateChange] The actual refactoring changes are constructed in this
+method.
+
+We iterate through every token in the current file to check if it has a comment
+string. Comment strings are acquired by calling \texttt{Token\#getWhiteBefore()}
+and \texttt{Token\#getWhiteAfter()}. Following the advice of
+Section~\ref{sec:refactoring_caveats}, we store a list of all the tokens (call
+this list TokensWithComments) that contain comment strings. Once we have
+iterated through all the tokens, we proceed to remove the comments for tokens in
+our TokensWithComments list. Removing comments is done by calling
+\texttt{Token\#setWhiteBefore()} and \texttt{Token\#setWhiteAfter()} with blank
+strings as parameters.
+
+Finally, we create a header comment that just lists all the functions and
+subroutines in the current source file and add that to the preamble of the main
+program.
+
+\end{description}
+
+For more information, please consult the source code for our example.
diff --git a/org.eclipse.photran-dev-docs/dev-guide/app-rename-tests.ltx-inc b/org.eclipse.photran-dev-docs/dev-guide/app-rename-tests.ltx-inc
index e158f6e1..3581254b 100644
--- a/org.eclipse.photran-dev-docs/dev-guide/app-rename-tests.ltx-inc
+++ b/org.eclipse.photran-dev-docs/dev-guide/app-rename-tests.ltx-inc
@@ -1,93 +1,122 @@
% Creating a Rename Refactoring Test Case
-\textit{Last Updated 4/12/07}
-
-\bfseries{\large This information is out of date and is being updated (8/07).}
-
JUnit tests for the Rename refactoring are located in the
-org.eclipse.photran.refactoring.tests plug-in. A Rename test
+\texttt{org.eclipse.photran.refactoring.tests} plug-in project. A Rename test
has two components:
+
\begin{enumerate}
+
\item one or more Fortran files, which contain the code to be refactored, and
-\item a JUnit test suite class, which creates tests attempting to rename the identifiers in that file.
+
+\item a JUnit test suite class, which creates tests attempting to rename the
+identifiers in that file.
+
\end{enumerate}
-The Fortran files are stored as .f90 files in the rename-test-code folder.
-The JUnit tests are similarly-named Java classes in the
+The Fortran files are stored as .f90 files in the \emph{rename-test-code}
+folder. The JUnit tests are similarly-named Java classes in the
\texttt{org.eclipse.photran.refactoring.tests.rename} package.
-A sample JUnit test suite is the following. The more complex tests follow a similar structure.
-Here, the \texttt{vars} array records all of the identifiers and the line/column positions on which they occur.
-The test suite constructor attempts to rename each identifier to \texttt{z} and also to
+A sample JUnit test suite is the following. The more complex tests follow a
+similar structure. Here, the \texttt{vars} array records all of the identifiers
+and the line/column positions on which they occur. The test suite constructor
+attempts to rename each identifier to \texttt{z} and also to
\texttt{a\_really\_really\_long\_name}.
-{\footnotesize\begin{verbatim}
+\shabox{Because our strategy for testing requires the \emph{exact} line and
+column position, using tabs instead of spaces for indenting could interfere with
+the positioning. Therefore, for testing purposes, test cases (the Fortran test
+files) should be indented with \textbf{spaces only}. However, when an actual
+Fortran programmer invokes a refactoring through the Eclipse UI, the indentation
+with tabs or spaces is not a problem because the Eclipse editor is smart enough
+to provide the correct position (based on expanding tabs into spaces
+internally). } \\
+
+\begin{lstlisting}[frame=lines]
public class Rename2 extends RenameTestSuite
{
- ///////////////////////////////////////////////////////////////////////////
- //
- // RECORD POSITIONS OF ALL IDENTIFIERS IN RENAME2.F90, AND
- // GROUP THEM ACCORDING TO WHICH ONES SHOULD BE RENAMED TOGETHER
- //
- ///////////////////////////////////////////////////////////////////////////
-
- private String filename = "rename2.f90";
-
- private Ident[] vars = new Ident[]
- {
- var(filename, "Main", new LineCol[] { lc(2,9), lc(27,13) }),
- var(filename, "one", new LineCol[] { lc(4,16), lc(12,14), lc(16,11), lc(20,11) }),
- var(filename, "two", new LineCol[] { lc(5,27), lc(10,13), lc(13,14), lc(17,14) }),
- var(filename, "three", new LineCol[] { lc(6,16), lc(14,9), lc(18,9) }),
- var(filename, "four", new LineCol[] { lc(10,21), lc(15,14), lc(19,14) })
- };
-
- ///////////////////////////////////////////////////////////////////////////
- //
- // TEST CASES
- //
- ///////////////////////////////////////////////////////////////////////////
-
- public static Test suite() throws Exception
- {
- return new Rename2();
- }
-
- public Rename2() throws Exception
- {
- startTests("Renaming program with comments and line continuations");
- for (String name : new String[] { "z", "a_really_really_long_name" })
- for (Ident var : vars)
- addSuccessTests(var, name);
- endTests();
- }
+ /////////////////////////////////////////////////////////////////////////
+ //
+ // RECORD POSITIONS OF ALL IDENTIFIERS IN RENAME2.F90, AND
+ // GROUP THEM ACCORDING TO WHICH ONES SHOULD BE RENAMED TOGETHER
+ //
+ /////////////////////////////////////////////////////////////////////////
+
+ private String filename = "rename2.f90";
+
+ private Ident[] vars = new Ident[]
+ {
+ var(filename, "Main", new LineCol[] { lc(2,9), lc(27,13) }),
+ var(filename, "one", new LineCol[] { lc(4,16), lc(12,14),
+ lc(16,11), lc(20,11) }),
+ var(filename, "two", new LineCol[] { lc(5,27), lc(10,13),
+ lc(13,14), lc(17,14) }),
+ var(filename, "three", new LineCol[] { lc(6,16), lc(14,9),
+ lc(18,9) }),
+ var(filename, "four", new LineCol[] { lc(10,21), lc(15,14),
+ lc(19,14) })
+ };
+
+ /////////////////////////////////////////////////////////////////////////
+ //
+ // TEST CASES
+ //
+ /////////////////////////////////////////////////////////////////////////
+
+ public static Test suite() throws Exception
+ {
+ return new Rename2();
+ }
+
+ public Rename2() throws Exception
+ {
+ startTests("Renaming program with comments and line continuations");
+ for (String name : new String[] { "z", "a_really_really_long_name" })
+ for (Ident var : vars)
+ addSuccessTests(var, name);
+ endTests();
+ }
}
-\end{verbatim}}
-
-The \texttt{addSuccessTests} method adds several test cases to the suite:
-it simulates the user clicking on each occurrence of the identifier
-and asking to rename that instance. (Of course, no matter which occurrence is clicked on,
-all instances should be renamed\ldots but this has occasionally not happened.)
-
-If the rename should not have succeeded---that is, a precondition would not be
-met---\texttt{addPreconditionTests} should have been
-called rather than \texttt{addSuccessTests}.
-
-\texttt{Rename3} is a slightly more complicated example, which renames identifiers spanning
-multiple files. In this case, a large boolean matrix is used to record which identifiers
-should be renamable to which other identifiers:
-{\footnotesize\begin{verbatim}
- private boolean[][] expectSuccess = new boolean[][]
- {
- // IMPORTANT:
- // * Modules can't be renamed, hence the rows of "false" for moduleA, moduleB, and moduleC
- // * Everything except myProgram and external should probably be renameable to myProgram, but this a corner case which we'll ignore
-
- /* vvv can be renamed to >>> myProgram, aRenamed3, bRenamed3, contained, external, ...
- /* myProgram */ new boolean[] { false, true, true, true, false, ...
- /* aRenamed3 */ new boolean[] { false, false, false, false, false, ...
- /* bRenamed3 */ new boolean[] { false, false, false, false, false, ...
- /* contained */ new boolean[] { false, false, false, false, false, ...
- /* external */ new boolean[] { false, false, false, false, false, ...
- ...
-\end{verbatim}}
+\end{lstlisting}
+
+The \texttt{addSuccessTests} method adds several test cases to the suite: it
+simulates the user clicking on each occurrence of the identifier and asking to
+rename that instance. (Of course, no matter which occurrence is clicked on, all
+instances should be renamed\ldots but this has occasionally not happened.)
+
+If the rename should not have succeeded--that is, a precondition would not be
+met--\texttt{addPreconditionTests} should have been called rather than
+\texttt{addSuccessTests}. A good testing strategy ensures that a program behaves
+correctly: it should do \textbf{only} what it is supposed to do and nothing
+more. In our case, it should rename only the identifiers that are affected and
+ensure that the other identifiers are left untouched.
+
+\texttt{Rename3} is a slightly more complicated example, which renames
+identifiers spanning multiple files. In this case, a large boolean matrix is
+used to record which identifiers should be renamable to which other identifiers:
+
+\begin{code}
+\begin{lstlisting}
+private boolean[][] expectSuccess = new boolean[][]
+{
+
+/* vvv can be renamed to >>> myProgram, aRenamed3, bRenamed3, contained,
+/* myProgram */ new boolean[] { false, true, true, true, ...
+/* aRenamed3 */ new boolean[] { true, false, false, false, ...
+/* bRenamed3 */ new boolean[] { true, false, false, false, ...
+/* contained */ new boolean[] { true, false, false, false, ...
+/* external */ new boolean[] { false, false, false, false, ...
+/* moduleA */ new boolean[] { false, false, false, false, ...
+/* aSub1of3 */ new boolean[] { true, false, false, false, ...
+/* aSub2of3 */ new boolean[] { true, false, false, false, ...
+/* aSub3of3 */ new boolean[] { true, true, true, true, ...
+/* moduleB */ new boolean[] { false, false, false, false, ...
+/* bSub1of3 */ new boolean[] { true, true, true, true, ...
+/* bSub2of3 */ new boolean[] { true, false, false, false, ...
+/* bSub3of3 */ new boolean[] { true, true, true, true, ...
+/* moduleC */ new boolean[] { false, false, false, false, ...
+/* cSub */ new boolean[] { true, true, true, true, ...
+};
+\end{lstlisting}
+\caption{Partial representation of the boolean matrix in \texttt{Rename3}}
+\end{code}
diff --git a/org.eclipse.photran-dev-docs/dev-guide/authors.ltx b/org.eclipse.photran-dev-docs/dev-guide/authors.ltx
index 0bacd182..b49bbbfb 100644
--- a/org.eclipse.photran-dev-docs/dev-guide/authors.ltx
+++ b/org.eclipse.photran-dev-docs/dev-guide/authors.ltx
@@ -1,4 +1,5 @@
% List of author names for the Photran developer's guide
+N. Chen \\
J. Overbey \\
%your name here \\
diff --git a/org.eclipse.photran-dev-docs/dev-guide/cdt.ltx-inc b/org.eclipse.photran-dev-docs/dev-guide/cdt.ltx-inc
index 9384f48c..47fae82e 100644
--- a/org.eclipse.photran-dev-docs/dev-guide/cdt.ltx-inc
+++ b/org.eclipse.photran-dev-docs/dev-guide/cdt.ltx-inc
@@ -1,3 +1,17 @@
+\section{Introduction}
+
+The C/C++ Development Tools (CDT)\footnote{See
+\url{http://www.eclipse.org/cdt/}} enables Eclipse to function as a first-class
+C/C++ IDE. CDT provides features that a programmer expects from an IDE such
+as project management, automated build, integrated debugging, etc. In addition,
+CDT also provides extension points for writing IDEs for other programming
+languages that follow the C/C++ edit-compile-debug-compile cycle closely;
+Fortran is one such language.
+
+Photran builds upon the CDT by leveraging its extension points. As such, it
+needs to follow certain conventions and expectations of the CDT. In this
+chapter, we discuss those conventions and expectations.
+
\section{CDT Terminology}
The following are CDT terms that will be used extensively when discussing
@@ -6,8 +20,8 @@ Photran.
\begin{itemize}
\item \textbf{Standard Make projects} are ordinary Eclipse projects, except that
-the CDT (and Photran) recognize them as being ``their own'' type of project (as
-opposed to, say, projects for JDT, EMF, or another Eclipse-based tool). The user
+CDT and Photran recognize them as being ``their own'' type of project (as
+opposed to, say, projects for JDT, EMF, or another Eclipse-based tool). Users
must supply their own Makefile, typically with targets ``clean'' and ``all.''
CDT/Photran cleans and builds the project by running \texttt{make}.
@@ -18,7 +32,7 @@ project. The \textbf{Managed Build System} is the part of CDT and Photran that
handles all of this.
\item \textbf{Binary parsers} are able to detect whether a file is a legal
-executable for a platform (and extract other information from it). The CDT
+executable for a platform (and extract other information from it). CDT
provides binary parsers for Windows (PE), Linux (ELF), Mac OS X (Mach), and
others. Photran does not provide any additional binary parsers.
@@ -26,26 +40,34 @@ others. Photran does not provide any additional binary parsers.
error parser, for example. Photran provides error parsers for Lahey Fortran, F,
g95, and others. Essentially, error parsers scan the output of \texttt{make} for
error messages for their associated compiler. When they see an error message
-they can recognize, they extract the filename, line number, and error message,
-and use it to populate the Problems view.
+that they recognize, they extract the filename, line number, and error message,
+and use it to populate the Problems View. See
+Appendix~\ref{ap:creating_an_error_parser} for an example on how to create an
+error parser.
\item CDT keeps a \textbf{model} of all of the files in a project. The model is
-essentially a tree of \textbf{elements}, which all derive from a (CDT Core)
+essentially a tree of \textbf{elements}, which all derive from the (CDT Core)
class \texttt{ICElement}. It is described in the next section.
\end{itemize}
\section{The Model}
+\emph{This section describes the CDT model in detail. Understanding the CDT
+model is useful for contributors who are interested in modifying the UI and how
+Fortran projects are managed in the Fortran perspective. Contributors who are
+interested in creating refactorings and program analysis tools should
+familiarize themselves with the Abstract Syntax Tree (AST) and Virtual Program
+Graph (VPG) described in Chapter~\ref{cha:parsing}.}
+
The Fortran Projects view in Photran is essentially a visualization of
the CDT's \textit{model}, a tree data structure describing the contents
of all Fortran Projects in the workspace as well as the high-level contents
(functions, aggregate types, etc.) of source files.
-Alain Magloire (CDT) described the model, A.K.A. the \texttt{ICElement}
-hierarchy, in the thread
-``Patch to create ICoreModel interface''
-on the cdt-dev mailing list:
+Alain Magloire (CDT) describes the model, A.K.A. the \texttt{ICElement}
+hierarchy, in the thread ``Patch to create ICoreModel interface'' on the cdt-dev
+mailing list:
\begin{verbatim}
So I'll explain a little about the ICElement and what we get out of it for
@@ -88,7 +110,9 @@ conditions) etc .... For a complete interface/view of the language, clients
should use the __AST__ interface.
\end{verbatim}
-From another one of Alain's posts in that thread:
+\pagebreak
+
+From another of Alain's posts in that thread:
\begin{verbatim}
Lets make sure we are on the same length about the ICElement hierarchy.
@@ -114,39 +138,124 @@ The CoreModel uses information taken from:
to build the hierarchy.
\end{verbatim}
+\shabox{The CDT model should \textbf{not} be confused with the Abstract Syntax
+Tree (AST) model that is discussed in Section~\ref{sec:ast_structure}. They are
+\textbf{not} identical. It is best to think of the CDT model as containing a
+\emph{partial/simplified view} of the AST model to represent the
+\emph{interesting} elements in the source code (program names, function names,
+subroutine names) \textbf{in addition} to a model of the current workspace
+resources (Fortran projects, Fortran source files, binary executables). \emph{In
+other words, the CDT model knows about the language and the resources}. The AST,
+on the other hand, completely models \emph{everything} in the source file (but
+nothing about the resources), including low-level elements that the user is
+unlikely to be interested in knowing about (assignment nodes, variable
+declarations). While low-level, these elements are useful for refactoring and
+program analysis.}
+
+By conforming to the CDT model, Photran is able to reuse various UI
+elements for \emph{free}. For instance, the Outline View for Photran is managed
+by CDT; Photran just needs to provide a CDT-compatible model to represent its
+project and source files.
+
+The \texttt{FortranLanguage} class is responsible for initializing concrete
+classes that will build up the model that CDT expects. For more
+information, refer to the FortranLanguage.java file in the
+\texttt{org.eclipse.photran.cdtinterface} plug-in project.
+
+There are \textbf{two} options for creating suitable \emph{model builders}:
+\begin{enumerate}
+\item The \texttt{org.eclipse.photran.cdtinterface} plug-in project defines the
+\\ \texttt{org.eclipse.photran.cdtinterface.modelbuilder} extension point that
+other plug-ins can extend. Plug-ins extending that extension point are
+responsible for providing a suitable model builder. Using this option, it is
+possible to have multiple model builders. The model builder to use can be
+selected in the workspace preferences (under Fortran $>$ CDT Interface).
+
+\item If there are no plug-ins that extend the
+\texttt{org.eclipse.photran.cdtinterface.modelbuilder} extension point, then
+Photran falls back on a default implementation. The default implementation is
+provided by the \texttt{SimpleFortranModelBuilder} class. It relies on simple
+lexical analysis to build the model. This simple model builder might not be able
+to accurately handle the more complicated features of the Fortran language.
+\end{enumerate}
+
+The Photran VPG (see Section~\ref{sec:virtual_program_graph}) inside the
+\texttt{org.eclipse.photran.core.vpg} project uses the first option to
+contribute a model builder. The relevant classes are under the
+\texttt{org.eclipse.photran.internal.core.model} package i.e.
+\texttt{FortranModelBuilder}, \texttt{FortranModelBuildingVisitor} and
+\texttt{FortranParseTreeModelBuildingVisitor}.
+
+As mentioned in the post by Alain, all model elements must implement the
+\texttt{ICElement} interface for CDT to recognize them. In Photran, the
+\texttt{FortranElement} class implements the \texttt{ICElement} interface and
+serves as the base class for all Fortran elements such as subroutines,
+functions, modules, variables, etc. Each subclass of \texttt{FortranElement}
+corresponds to an element that can be displayed in the Outline View.
+
+\section{Reusing UI Elements}
+
+Various UI elements in Photran are also reused from the CDT through subclassing.
+For instance, the \texttt{NewProjectDropDownAction} class shown in
+Listing~\ref{lst:newprojectdropdownaction} is a subclass of
+\texttt{AbstractWizardDropDownAction} declared in CDT.
+\texttt{AbstractWizardDropDownAction} provides most of the implementation and
+our subclass just provides the Photran-specific details such as the actual
+action that will be invoked.
+
+\begin{code}
+\begin{lstlisting}
+public class NewProjectDropDownAction extends AbstractWizardDropDownAction
+{
+ public NewProjectDropDownAction()
+ {
+ super();
+ PlatformUI.getWorkbench().getHelpSystem().setHelp(this,
+ ICHelpContextIds.OPEN_PROJECT_WIZARD_ACTION);
+ }
+
+ protected IAction[] getWizardActions()
+ {
+ return FortranWizardRegistry.getProjectWizardActions();
+ }
+}
+\end{lstlisting}
+\caption{\texttt{NewProjectDropDownAction} class}
+\label{lst:newprojectdropdownaction}
+\end{code}
+
+Our \texttt{NewProjectDropDownAction} is invoked through the right-click menu by
+going to New... $>$ Other $>$ Fortran. It creates a new Fortran project in the
+current workspace.
+
+In addition, we could also customize the icons for each UI element by modifying
+the appropriate attributes in the plugin.xml file in the
+\texttt{org.eclipse.photran.cdtinterface} project.
+
\section{The CDT Debugger and \texttt{gdb}}
+Currently, Photran re-uses the CDT debugger as-is and does not contribute any
+enhancements to it. Here is a brief summary of how the debugger works:
+
\begin{itemize}
\item The so-called CDT debugger is actually just a graphical interface to
-\texttt{gdb}, or more specifically to \texttt{gdb/mi}. If something
-doesn't work, try it in \texttt{gdb} directly, or using another
-\texttt{gdb}-based tool such as DDD.
+\texttt{gdb}, or more specifically to \texttt{gdb/mi}. So, if something doesn't
+appear to work, it is advisable to try it in \texttt{gdb} directly or to use
+another \texttt{gdb}-based tool such as DDD.
-\item The debugger UI ``contributes'' breakpoint markers and actions
-to the editor. The ``set breakpoint'' action, and the breakpoint markers
-that appear in the ruler of the CDT (and Photran) editors are handled
-entirely by the debug UI: You will \textit{not} find code for them in the
-Photran UI plug-in.
+\item The debugger UI ``contributes'' breakpoint markers and actions to the
+editor. The ``set breakpoint'' action, and the breakpoint markers that appear in
+the ruler of the CDT (and Photran) editors are handled \textbf{entirely} by the
+CDT debug UI; there is no code for this in Photran. The ``set breakpoint''
+action is enabled by calling \lstinline!setRulerContextMenuId("#CEditorRulerContext");!
+in the \texttt{AbstractFortranEditor} constructor.
\item \texttt{gdb} reads debug symbols from the executable it is debugging.
That is how it knows what line it's on, what file to open, etc. Photran
has \textit{nothing} to do with this: These symbols are written entirely
by the compiler. Moreover, the compiler determines what shows up in the
-Variables view. If the debugger views seem to be a mess, it is the
+Variables View. If the debugger views seem to be a mess, it is the
compiler's fault, not Photran's.
\end{itemize}
-
-% TODO: Stuff to add-in from Jeff's e-mail
-
-% Contributed to CDT:
-% FortranLanguage (contributed to org.eclipse.cdt.core.language)
-% Contributes a model builder and possibly a DOM parser to CDT. Note that these are contributed to extension points defined in org.eclipse.photran.cdtinterface, which in turn contributes them to org.eclipse.cdt.core.language via the FortranLanguage class.
-% Reused from CDT:
-% Some UI reused through copying (no better alternative): FortranPerspectiveFactory, large parts of org.eclipse.photran.cdtinterface's plugin.xml
-% Lots of UI reused through subclassing: FortranView, NewFileDropDownAction, and many others
-% FortranElement (base class for elements in Photran's model) subclasses from CDT model classes
-% Some portions of the editor reused from CDT. Would have to read the code to remember the details.
-% I'm not sure about Managed Make.
-% The debugger is reused as-is. The user is able to set breakpoints in the Fortran editor because we reuse the C Editor's ruler context (see AbstractFortranEditor ctor).
diff --git a/org.eclipse.photran-dev-docs/dev-guide/cvs-instructions.ltx b/org.eclipse.photran-dev-docs/dev-guide/cvs-instructions.ltx
index 984cbf3b..40ee4941 100644
--- a/org.eclipse.photran-dev-docs/dev-guide/cvs-instructions.ltx
+++ b/org.eclipse.photran-dev-docs/dev-guide/cvs-instructions.ltx
@@ -2,7 +2,14 @@
\usepackage{anysize}
\marginsize{1.5in}{1.5in}{1.5in}{1.5in}
-%\usepackage{fullpage}
+\usepackage[pdftex]{hyperref}
+\hypersetup{
+ colorlinks,%
+ citecolor=black,%
+ filecolor=black,%
+ linkcolor=black,%
+ urlcolor=blue
+}
\hyphenpenalty=5000
\tolerance=1000
diff --git a/org.eclipse.photran-dev-docs/dev-guide/cvs-instructions.pdf b/org.eclipse.photran-dev-docs/dev-guide/cvs-instructions.pdf
index 03adf7b7..bd2a6346 100644
--- a/org.eclipse.photran-dev-docs/dev-guide/cvs-instructions.pdf
+++ b/org.eclipse.photran-dev-docs/dev-guide/cvs-instructions.pdf
Binary files differ
diff --git a/org.eclipse.photran-dev-docs/dev-guide/dev-guide.ltx b/org.eclipse.photran-dev-docs/dev-guide/dev-guide.ltx
index b3b38c0b..295cf048 100644
--- a/org.eclipse.photran-dev-docs/dev-guide/dev-guide.ltx
+++ b/org.eclipse.photran-dev-docs/dev-guide/dev-guide.ltx
@@ -3,11 +3,21 @@
%\usepackage{anysize}
%\marginsize{0.5in}{0.5in}{0.5in}{0.5in}
+
+
+% Unnumbered footnotes
+\long\def\symbolfootnote[#1]#2{\begingroup\def\thefootnote{\fnsymbol{footnote}}
+\footnote[#1]{#2}\endgroup}
+
\usepackage{fullpage}
-\usepackage{graphicx}
+\usepackage[pdftex]{graphicx}
+\usepackage{color}
+\definecolor{LightMagenta}{cmyk}{0.1,0.8,0,0.1}
\usepackage{listings}
-\lstset{numbers=left, numberstyle=\tiny, numbersep=5pt, captionpos=b, basicstyle=\footnotesize}
+\lstset{numbers=left, numberstyle=\tiny, tabsize=2, numbersep=5pt, captionpos=b,
+basicstyle=\footnotesize, showstringspaces=false,
+emphstyle=\color{LightMagenta}\bfseries, language=Java}
\usepackage[pdftex]{hyperref}
\hypersetup{
@@ -18,7 +28,15 @@
urlcolor=blue
}
+\usepackage{float}
+\floatstyle{ruled}
+\newfloat{image}{htp}{lop}[chapter]
+\floatname{image}{Figure}
+
+\newfloat{code}{htp}{lop}[chapter]
+\floatname{code}{Listing}
+\usepackage{shadow}
\hyphenpenalty=5000
\tolerance=1000
@@ -41,36 +59,48 @@
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
-\chapter{Introduction}
+\chapter{Introduction}\symbolfootnote[0]{Last significant update: 08/08/08}
+\label{cha:introduction}
\input{intro.ltx-inc}
-\chapter{Interactions with the CDT}
+\chapter{Interactions with CDT}\symbolfootnote[0]{Last significant update: 08/08/08}
+\label{cha:cdt}
\input{cdt.ltx-inc}
-\chapter{Plug-in Decomposition}
+\chapter{Plug-in Decomposition}\symbolfootnote[0]{Last significant update: 08/08/08}
+\label{cha:plugins}
\input{plugins.ltx-inc}
-\chapter{Parsing and Program Analysis}
+\chapter{Parsing and Program Analysis}\symbolfootnote[0]{Last significant update: 08/08/08}
+\label{cha:parsing}
\input{parsing.ltx-inc}
-\chapter{Refactoring}
+\chapter{Refactoring}\symbolfootnote[0]{Last significant update: 08/08/08}
+\label{cha:refactoring}
\input{refactoring.ltx-inc}
-\chapter{Photran Editor}
+\chapter{Photran Editors}\symbolfootnote[0]{Last significant update: 08/08/08}
+\label{cha:editor}
\input{editor.ltx-inc}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\appendix
-\chapter{Getting the Photran 4.0 Sources from CVS}
-\vspace{0.5em}
+\chapter{Getting the Photran 4.0 Sources from CVS}\symbolfootnote[0]{Last significant update: 08/08/08}
\input{app-cvs.ltx-inc}
-\chapter{Creating an Error Parser}
+\chapter{Creating an Error Parser}\symbolfootnote[0]{Last significant update: 08/08/08}
+\label{ap:creating_an_error_parser}
\input{app-error-parsers.ltx-inc}
-\chapter{Creating Tests for the Rename Refactoring}
+\chapter{Simple Fortran Refactoring Example}\symbolfootnote[0]{Last significant update: 08/08/08}
+\label{app:obfuscate_refactoring}
+\input{app-obfuscate-fortran.ltx-inc}
+
+\chapter{Creating Tests for the Rename Refactoring}\symbolfootnote[0]{Last significant update: 08/08/08}
\input{app-rename-tests.ltx-inc}
+
+
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\end{document}
diff --git a/org.eclipse.photran-dev-docs/dev-guide/editor.ltx-inc b/org.eclipse.photran-dev-docs/dev-guide/editor.ltx-inc
index 6a82ade8..28d52995 100644
--- a/org.eclipse.photran-dev-docs/dev-guide/editor.ltx-inc
+++ b/org.eclipse.photran-dev-docs/dev-guide/editor.ltx-inc
@@ -1 +1,153 @@
-Chapter to describe the Photran editor. We will focus on describing how the VPG editor actions work here.
+\section{Fortran Text Editors}
+
+There are \textbf{two} different text editors in Photran. This is necessary to
+support both the fixed-form Fortran 77 standard and the free-form Fortran 90 \&
+95 standard.
+
+Fortran 77 is known as fixed-form Fortran because it requires certain constructs
+to be \emph{fixed} to particular columns. For instance, Fortran statements can
+only appear between columns 7 - 72; anything beyond column 72 is ignored
+completely. This requirement is an artifact of the days when punched cards were
+used for Fortran programming. However, Fortran 77 compilers still enforce this
+requirement. The fixed-form editor in Photran helps the programmer remember this
+requirement by displaying visual cues to denote the column partitions.
+
+Fortran 90/95 adopted the free-form format that most programmers today are
+accustomed to. Nonetheless, because Fortran 77 is still considered a subset of
+Fortran 90/95, it is possible to write programs following the fixed-form format.
+As such, the free-form editor maintains some visual cues on column numbering
+(although using a more subtle UI).
+
+The UML class diagram in Figure~\ref{fig:images_editor_class_hierarchy} shows
+the hierarchy of the editors in Photran.
+
+\begin{image}
+ \centering
+ \includegraphics[height=3in]{images/editor_hierarchy.png}
+ \caption{Photran editor class hierarchy}
+ \label{fig:images_editor_class_hierarchy}
+\end{image}
+
+Both the \texttt{FixedFormFortranEditor} and \texttt{FreeFormFortranEditor}
+concrete classes inherit from \texttt{AbstractFortranEditor}. Most of the actual
+work is done inside \texttt{AbstractFortranEditor}; its subclasses just specify
+how to decorate the UI.
+
+In general, the implementation of \texttt{AbstractFortranEditor} closely follows
+the standard implementation of text editors in Eclipse. The following section
+highlights some of the Photran-specific mechanisms of the text editor. For more
+information on how text editors work in Eclipse, please consult the Eclipse
+references mentioned in Chapter~\ref{cha:introduction}.
+
+\section{Contributed \texttt{SourceViewerConfiguration}}
+
+Text editors in Eclipse rely on a \texttt{SourceViewerConfiguration} to enhance
+the current editor with features such as auto indenting, syntax highlighting and
+formatting. By default, most of these features are already provided by the
+concrete \texttt{SourceViewerConfiguration} class. However, it is possible to
+provide a custom implementation of a \texttt{SourceViewerConfiguration}. This is
+done by calling the
+\texttt{setSourceViewerConfiguration(SourceViewerConfiguration
+sourceViewerConfiguration)} method in an Eclipse text editor.
+
+Photran provides an additional layer of flexibility by allowing its
+\texttt{SourceViewerConfiguration} to be contributed from other plug-ins. A
+plug-in that is interested in contributing a \texttt{SourceViewerConfiguration}
+to the Photran editors must extend the
+\\\texttt{org.eclipse.photran.ui.sourceViewerConfig} extension point defined in
+the \\\texttt{org.eclipse.photran.ui.vpg} plug-in.
+
+At run-time, Photran \emph{dynamically} instantiates a suitable
+\texttt{SourceViewerConfiguration} by searching through the list of plug-ins
+that extend the \texttt{org.eclipse.photran.ui.sourceViewerConfig} extension.
+
+Currently, there are \textbf{two} \texttt{SourceViewerConfiguration}s in
+Photran: the contributed \texttt{FortranVPGSourceViewerConfigurationFactory} and
+the default (but less feature-full)
+\texttt{FortranModelReconcilingSourceViewerConfiguration}.
+
+\section{Fortran Editor Tasks: VPG \& AST Tasks}
+
+Many actions that a user can invoke actually depend on the current text in the
+text editor. For instance, auto-completion depends on the text that has been
+entered so far to provide feasible completion choices. A good auto-completion
+strategy constantly augments and updates its database of feasible completion
+choices as the user enters or modifies text; it does not have to wait until the
+user saves the current file.
+
+Another important feature of the text editor that requires constant updates is
+syntax highlighting. Syntax highlighting has to work almost instantaneously
+based on what the user has entered. It is not acceptable for the user to
+experience lengthy delays between typing a character and waiting for it to
+syntax highlight correctly.
+
+Eclipse utilizes a \emph{reconciler} to correctly and instantly perform syntax
+highlighting. The reconciler runs in a background thread in Eclipse, constantly
+monitoring the text that the user enters and updating the syntax highlighting as
+necessary. Every text editor in Eclipse -- including Photran's -- has a
+corresponding reconciler.
+
+Photran takes advantage of its existing reconciler
+(\texttt{FortranVPGReconcilingStrategy}) and adds additional Fortran editor
+tasks that should run each time its reconciler runs. The list of tasks to run is
+stored in the singleton \texttt{FortranEditorTasks} object.
+
+Currently, there are two kinds of tasks that can be run: Abstract Syntax Tree
+(AST) editor tasks and Virtual Program Graph(VPG) editor tasks. AST editor tasks
+depend on information from the AST of the current source file; and VPG editor
+tasks depend on information from the VPG of the current source file.
+\texttt{FortranEditorTasks} automatically schedules the VPG editor tasks using
+an instance of \texttt{VPGSchedulingRule} to synchronize access to the
+\texttt{PhotranVPG} singleton object. The AST of the current file is computed
+on-the-fly as the user modifies the source file. The VPG of the current file is
+based off its previous saved version (so it is less up-to-date). For more
+information about the AST and VPG, see Chapter~\ref{cha:parsing}.
+
+AST editor tasks must implement the \texttt{IFortranEditorASTTask} interface and
+VPG editor tasks must implement the \texttt{IFortranEditorVPGTask} interface.
+Additionally, each task has to register itself with the
+\texttt{FortranEditorTasks} object. A task that no longer needs to run should
+also be unregistered. Since these tasks run asynchronously, it is important to
+use proper Java concurrency practices i.e. \textbf{synchronized} methods and statements.
+
+\pagebreak
+
+Below is the API of the \texttt{FortranEditorTasks} class:
+
+\begin{code}
+\begin{lstlisting}
+public class FortranEditorTasks
+{
+ public static FortranEditorTasks instance(AbstractFortranEditor editor)
+
+ public synchronized void addASTTask(IFortranEditorASTTask task)
+
+ public synchronized void addVPGTask(IFortranEditorVPGTask task)
+
+ public synchronized void removeASTTask(IFortranEditorASTTask task)
+
+ public synchronized void removeVPGTask(IFortranEditorVPGTask task)
+
+ public Runner getRunner()
+
+ ...
+}
+\end{lstlisting}
+\caption{API of \texttt{FortranEditorTasks} (see FortranEditorTasks.java)}
+\end{code}
+
+It is possible for a class to implement both the \texttt{IFortranEditorASTTask}
+and \texttt{IFortranEditorVPGTask} interfaces. For example, the
+\texttt{DeclarationView} class registers itself for both kinds of editor tasks
+and makes use of the information from both as it attempts to present the
+declaration for the currently selected token of the text editor.
+
+For more information on implementation details, please refer to the following
+classes:
+
+\begin{itemize}
+ \item \texttt{DeclarationView}
+ \item \texttt{FortranCompletionProcessorASTTask}
+ \item \texttt{FortranCompletionProcessorVPGTask}
+ \item \texttt{OpenDeclarationASTTask}
+\end{itemize}
diff --git a/org.eclipse.photran-dev-docs/dev-guide/images/diagrams/editor_hierarchy.graffle b/org.eclipse.photran-dev-docs/dev-guide/images/diagrams/editor_hierarchy.graffle
new file mode 100644
index 00000000..4847a60f
--- /dev/null
+++ b/org.eclipse.photran-dev-docs/dev-guide/images/diagrams/editor_hierarchy.graffle
Binary files differ
diff --git a/org.eclipse.photran-dev-docs/dev-guide/images/diagrams/parser_chain.graffle b/org.eclipse.photran-dev-docs/dev-guide/images/diagrams/parser_chain.graffle
new file mode 100644
index 00000000..5c407204
--- /dev/null
+++ b/org.eclipse.photran-dev-docs/dev-guide/images/diagrams/parser_chain.graffle
Binary files differ
diff --git a/org.eclipse.photran-dev-docs/dev-guide/images/editor_hierarchy.png b/org.eclipse.photran-dev-docs/dev-guide/images/editor_hierarchy.png
new file mode 100644
index 00000000..22fce3a1
--- /dev/null
+++ b/org.eclipse.photran-dev-docs/dev-guide/images/editor_hierarchy.png
Binary files differ
diff --git a/org.eclipse.photran-dev-docs/dev-guide/images/parser_chain.png b/org.eclipse.photran-dev-docs/dev-guide/images/parser_chain.png
new file mode 100644
index 00000000..d70d8aa7
--- /dev/null
+++ b/org.eclipse.photran-dev-docs/dev-guide/images/parser_chain.png
Binary files differ
diff --git a/org.eclipse.photran-dev-docs/dev-guide/images/simple_fortran_AST.png b/org.eclipse.photran-dev-docs/dev-guide/images/simple_fortran_AST.png
new file mode 100644
index 00000000..5c27826f
--- /dev/null
+++ b/org.eclipse.photran-dev-docs/dev-guide/images/simple_fortran_AST.png
Binary files differ
diff --git a/org.eclipse.photran-dev-docs/dev-guide/intro.ltx-inc b/org.eclipse.photran-dev-docs/dev-guide/intro.ltx-inc
index 3c3b1af3..dba7b3a4 100644
--- a/org.eclipse.photran-dev-docs/dev-guide/intro.ltx-inc
+++ b/org.eclipse.photran-dev-docs/dev-guide/intro.ltx-inc
@@ -1,39 +1,40 @@
% Introduction
-
-\textit{Last Updated 08/01/07}
+\vspace{-0.5in}
Photran is an IDE for Fortran 90/95 and Fortran 77 that is built on top of
Eclipse. It is structured as an Eclipse feature, in other words, as a set of
plug-ins that are designed to be used together. Starting with version 3.0, it is
-an extension of CDT, the Eclipse IDE for C/C++. Previous versions of Photran
-were created by hacking a copy of the CDT to support Fortran instead of C/C++,
-but now we have developed a mechanism for adding new languages into the CDT,
-allowing the Fortran support code to be in its own set of plug-ins.
+an extension of C/C++ Development Tools (CDT), the Eclipse IDE for C/C++.
+Previous versions of Photran were created by hacking a copy of CDT to support
+Fortran instead of C/C++, but now we have developed a mechanism for adding new
+languages into CDT, allowing the Fortran support code to be in its own set of
+plug-ins.
Our purpose in writing Photran was to create a refactoring tool for Fortran.
Thus, Photran has a complete parser and program representation. Photran adds a
-Fortran editor and several preference pages to the CDT user interface, as well
-as a Fortran Managed Make project type.
+Fortran editor and several preference pages to the CDT user interface, as
+well as a Fortran Managed Make project type.
+
+\vspace{-0.2in}
-\section{How To Read This Guide} % (fold)
-\label{sec:how_to_read_this_guide}
+\section{How To Read This Guide}
This document explains the design of Photran so that interested contributors
could fix a bug or add a refactoring. Contributors should know how to use
-Photran and how the CDT works. There is a short
+Photran and how CDT works. There is a short
\href{http://www.eclipse.org/photran/documentation.php}{\emph{Getting Started
Guide}} on the Photran website.
-Contributors also need to understand Eclipse and Eclipse plug-ins before you
-read this document. We recommend reading
+Contributors also need to understand Eclipse and how to build Eclipse plug-ins
+before they read this document. We recommend reading
\href{http://www.amazon.com/Eclipse-Building-Commercial-Quality-Plug-Ins/dp/0321228472}{\emph{Building
Commercial-Quality Plug-ins}} and
\href{http://www.amazon.com/Java-Developers-Guide-Eclipse-2nd/dp/0321305027/}{\emph{The
Java Developer's Guide to Eclipse}} for Eclipse newcomers.
This is a \emph{duplex} guide. The main chapters provide general descriptions of
-the various components and how they interact. The appendix describe concrete
+the various components and how they interact. The appendix describes concrete
examples from Photran so that contributors can familiarize themselves with
-actual code and implementation details.
-
-% section how_to_read_this_guide (end)
+actual code and implementation details. This guide complements the source code
+in the repository; it is \underline{not a substitute} for reading the actual
+source code. \ No newline at end of file
diff --git a/org.eclipse.photran-dev-docs/dev-guide/parsing.ltx-inc b/org.eclipse.photran-dev-docs/dev-guide/parsing.ltx-inc
index cad93b0d..583d7d6b 100644
--- a/org.eclipse.photran-dev-docs/dev-guide/parsing.ltx-inc
+++ b/org.eclipse.photran-dev-docs/dev-guide/parsing.ltx-inc
@@ -1,262 +1,657 @@
% Parsing and Program Analysis
-\textit{Last Updated 10/2/07}
-
-\section{The Virtual Program Graph}
+\section{Parsing}
+
+Before any program analysis can be done, the source code of the Fortran program
+has to be parsed. Photran provides support for both fixed-form (Fortran 77) and
+free-form Fortran (Fortran 90 \& 95) source code. The parser in Photran is
+generated using the Ludwig parser/AST generator by Jeff Overbey. It is based on
+the \emph{fortran95.bnf} grammar file.
+
+Figure~\ref{fig:images_parser_chain} shows the lexer \& parser tool chain in
+Photran. Preliminary support for the preprocessor (\texttt{INCLUDE} directives)
+has also been implemented in the current version of the tool chain.
+
+\begin{image}
+ \centering
+ \includegraphics[height=3in]{images/parser_chain.png}
+ \caption{Photran preprocessor/lexer/parser chain}
+ \label{fig:images_parser_chain}
+\end{image}
+
+\section{Virtual Program Graph}
+\label{sec:virtual_program_graph}
+
+% I can't seem to get the cedilla fa\c{c}ade to render in my version of LaTeX
+In Photran, it is \emph{almost} never necessary to call the lexer, parser, or
+analysis components directly. Instead, Photran uses a \textbf{virtual program
+graph} (VPG), which provides the facade of a whole-program abstract syntax tree
+(AST) with embedded analysis information. In general, program analysis should be
+performed through the Photran VPG.
+
+An overview of VPGs is available at the following link:
+\url{http://jeff.over.bz/software/vpg/doc/}. It provides the background
+necessary for the remaining sections of this chapter.
+
+\subsection{Using the Photran VPG}
+
+\subsubsection{Acquiring and Releasing Translation Units}
+% TODO Make sure that this fits nicely into the page
+\texttt{PhotranVPG} is a \emph{singleton} object responsible for constructing
+ASTs for Fortran source code. ASTs are retrieved by invoking either of these
+methods:
+\begin{code}
+\begin{lstlisting}[numbers=none]
+public IFortranAST acquireTransientAST(IFile file)
-In Photran, it is almost never necessary to call the lexer, parser, or analysis components directly. Instead, it
-uses a \textit{virtual program graph} (VPG), which provides the fa\,cade of a whole-program abstract syntax tree with
-embedded analysis information.
+public IFortranAST acquirePermanentAST(IFile file)
+\end{lstlisting}
+\caption{Acquiring the Fortran AST}
+\end{code}
+
+\vspace{-0.15in}
+
+The returned object is an IFortranAST, an object which has a method for
+returning the root node of the AST as well as methods to quickly locate tokens
+in the AST by offset or line information. A \emph{transient AST} can be garbage
+collected as soon as references to any of its nodes disappear. A \emph{permanent
+AST} will be explicitly kept in memory until a call is made to either of the
+following methods:
+\begin{code}
+\begin{lstlisting}[numbers=none]
+public void releaseAST(IFile file)
-An overview of VPGs is available at the following link and will not repeated here:
+public void releaseAllASTs()
+\end{lstlisting}
+\caption{Releasing the Fortran AST}
+\end{code}
+
+Often, it is better to acquire a transient AST and rely on the garbage collector
+to reclaim the memory once we are done using it. However, there are times when
+acquiring a permanent AST would be more beneficial performance-wise. For
+instance, if we will be using the same AST multiple times during a refactoring,
+it would be better to just acquire a permanent AST. This prevents the garbage
+collector from reclaiming the memory midway through the refactoring once all
+references to the AST have been invalidated. While it is always possible to
+reacquire the same AST, doing so can be an expensive operation since it requires
+\emph{lexing}, \emph{parsing} \textbf{and} finally \emph{reconstructing} the AST
+from scratch.
-http://jeff.over.bz/software/vpg/doc/
+Only one AST for a particular file is in memory at any particular point in time,
+so successive requests for the same \texttt{IFile} will return the same
+(pointer-identical) AST until the AST is released (permanent) or garbage
+collected (transient).
+
+\subsubsection{Caveat}
+
+It is important to note that, because \texttt{PhotranVPG} is a singleton object,
+one must be careful about accessing it from multiple threads. Most of the time,
+when an AST is required, it will be from an action contributed to the Fortran
+editor; in this case, the action will usually be a descendant of
+\texttt{FortranEditorActionDelegate}, and synchronization will be handled
+automatically. For instance, all refactoring actions in Photran are descendants
+of \texttt{FortranEditorActionDelegate} and their accesses to
+\texttt{PhotranVPG} are being synchronized automatically.
+
+Otherwise, the thread must either be scheduled using a
+\texttt{VPGSchedulingRule} or it must lock the entire workspace . See
+\texttt{EclipseVPG\#queueJobToEnsureVPGIsUpToDate} as an example on how to use
+the \texttt{VPGSchedulingRule} and \texttt{FortranEditorActionDelegate\#run} as
+an example of how to lock the entire workspace.
+
+As a guideline, contributors who are interested in accessing the VPG should
+consider structuring their contributions as descendants of
+\texttt{FortranEditorActionDelegate} since that is the simplest mechanism (all
+synchronization is already taken care of automatically). However, if such an
+approach is not feasible, then they should consider using
+\texttt{VPGSchedulingRule} before resorting to locking the entire workspace.
-\section{Using the Photran VPG}
+\subsection{The Program Representation: IFortranAST}
-\subsection{Acquiring and Releasing Translation Units}
+The \texttt{acquireTransientAST} and \texttt{acquirePermanentAST} methods return
+an object implementing \texttt{IFortranAST}. Listing~\ref{lst:ifortranast} shows
+the methods in IFortranAST.
-\texttt{PhotranVPG} is a singleton object responsible for constructing
-ASTs for Fortran source code. ASTs are retrieved by invoking either
-{\footnotesize\begin{verbatim}
- public IFortranAST acquireTransientAST(IFile file)
-\end{verbatim}}
-or
-{\footnotesize\begin{verbatim}
- public IFortranAST acquirePermanentAST(IFile file)
-\end{verbatim}}
+\begin{code}
+\begin{lstlisting}[firstnumber=25]
+public interface IFortranAST extends Iterable<Token>
+{
+ ///////////////////////////////////////////////////////////////////////////
+ // Visitor Support
+ ///////////////////////////////////////////////////////////////////////////
+
+ public void accept(IASTVisitor visitor);
+
+ ///////////////////////////////////////////////////////////////////////////
+ // Other Methods
+ ///////////////////////////////////////////////////////////////////////////
+
+ public ASTExecutableProgramNode getRoot();
+
+ public Iterator<Token> iterator();
+ public Token findTokenByStreamOffsetLength(int offset, int length);
+ public Token findFirstTokenOnLine(int line);
+ public Token findTokenByFileOffsetLength(IFile file, int offset, int length);
+}
+\end{lstlisting}
+\caption{\texttt{IFortranAST} (see IFortranAST.java)}
+\label{lst:ifortranast}
+\end{code}
+
+The \texttt{getRoot} method returns the root of the AST, while the
+\texttt{find...} methods provide efficient means to search for tokens based
+on their lexical positioning in the source code.
+
+The \texttt{accept} method allows an external visitor to traverse the AST. This
+method is usually used when it is necessary to ``collect'' information about
+certain nodes. For more information see Section~\ref{sec:ast_structure} on what
+nodes can be visited.
+
+Because \texttt{IFortranAST} extends the \texttt{Iterable} interface, it is
+possible to use the \emph{foreach} loop to conveniently iterate through all the
+tokens in the AST e.g. \\ \lstinline!for (Token token : new IterableWrapper<Token>(ast))!
+
+\section{AST Structure}
+\label{sec:ast_structure}
+
+Photran's (rewritable) AST is generated along with the parser, so the structure
+of an AST is determined by the structure of the parsing grammar (see the
+\emph{fortran95.bnf} file). The generated classes are located in the
+\texttt{org.eclipse.photran.internal.core.parser} package in the
+\texttt{org.eclipse.photran.core.vpg} project. The easiest way to visualize the
+structure of a particular file's AST is to view it in the Outline view (see
+Section~\ref{sec:how_to_get_acquainted_with_the_program_representation}).
+However determining all possible ASTs for a particular construct requires
+scrutinizing the parsing grammar file.
+
+\subsection{Ordinary AST Nodes}
+\label{sub:ordinary_ast_nodes}
+
+Generally speaking, there is one AST node for each nonterminal in the grammar
+and one accessor method for each symbol on its right-hand side (unless the
+symbol name is prefixed with a hyphen, in which case it is omitted). For
+example, the following specification\footnote{All grammar
+specifications are taken from the \emph{fortran95.bnf} file. The \# RXXX number
+provides a reference to the actual specification in the grammar file.}
-A \textit{transient AST} can be garbage collected as soon as references to any
-of its nodes disappear. A \textit{permanent AST} will be explicitly kept in
-memory until a call is made to either
{\footnotesize\begin{verbatim}
- public void releaseAST(IFile file)
+# R533
+<DataStmtSet> ::= <DataStmtObjectList> -:T_SLASH <DataStmtValueList> -:T_SLASH
\end{verbatim}}
-or
-{\footnotesize\begin{verbatim}
- public void releaseAllASTs()
-\end{verbatim}}
-
-Only one AST for a particular file is in memory at any particular point in time,
-so successive requests for the same
-\texttt{IFile} will return the same (pointer-identical) AST
-until the AST is released (permanent) or garbage collected (transient).
-\subsection{The Program Representation: IFortranAST}
+generates the AST node class \texttt{ASTDataStmtSetNode} shown in
+Listing~\ref{lst:astdatastmtsetnode_api}. Notice the \emph{presence} of the
+\texttt{getDataStmtObjectList} and \texttt{getDataStmtValueList} getters methods
+and the \emph{absence} of any method for accessing T\_SLASH.
-The \texttt{acquireTransient/PermanentAST} methods return an object implementing \texttt{IFortranAST}.
+\shabox{The convention is to generate a class with the name
+\texttt{\color{LightMagenta}AST}$<$nonterminal
+name$>$\texttt{\color{LightMagenta}Node} that extends \texttt{ASTNode}. For
+instance \# R533 will generate the
+\texttt{{\color{LightMagenta}AST}DataStmtSet{\color{LightMagenta}Node}} class.}
-\lstset{language=Java}
+The following sections describe additional annotations that can be used to
+modify the standard convention when necessary. These annotations are not
+considered part of the standard BNF notation but they are supported by the
+underlying Ludwig parser generator.
+\begin{code}
\begin{lstlisting}
-public interface IFortranAST extends Iterable<Token>
+public class ASTDataStmtSetNode extends ASTNode
{
- ///////////////////////////////////////////////////////////////////////////
- // Visitor Support
- ///////////////////////////////////////////////////////////////////////////
-
- public void visitTopDownUsing(ASTVisitor visitor);
- public void visitBottomUpUsing(ASTVisitor visitor);
- public void visitOnlyThisNodeUsing(ASTVisitor visitor);
- public void visitUsing(GenericParseTreeVisitor visitor);
-
- ///////////////////////////////////////////////////////////////////////////
- // Other Methods
- ///////////////////////////////////////////////////////////////////////////
-
- public ASTExecutableProgramNode getRoot();
-
- public Iterator<Token> iterator();
- public Token findTokenByStreamOffsetLength(int offset, int length);
- public Token findFirstTokenOnLine(int line);
- public Token findTokenByFileOffsetLength(IFile file, int offset, int length);
+ public IASTListNode<IDataStmtObject> getDataStmtObjectList() {...}
+
+ public void setDataStmtObjectList(IASTListNode<IDataStmtObject> newValue) {...}
+
+ public IASTListNode<ASTDataStmtValueNode> getDataStmtValueList() {...}
+
+ public void setDataStmtValueList(IASTListNode<ASTDataStmtValueNode> newValue) {...}
+
+ ...
}
\end{lstlisting}
+\caption{\texttt{ASTDataStmtSetNode} generated from \# R533}
+\label{lst:astdatastmtsetnode_api}
+\end{code}
+
+\subsubsection{Annotation \#1: \texttt{(list)}}
+
+Recursive productions are treated specially, since they are used frequently to
+express lists in the grammar. The recursive member is labeled in the grammar
+with the \texttt{(list)} annotation. For example, the following specification
-The \texttt{getRoot} method returns the root of the AST, while the \texttt{find...}
-methods provide an efficient means to search for tokens based on their lexical
-positioning.
+{\footnotesize\begin{verbatim}
+# R538
+(list):<DataStmtValueList> ::=
+ | <DataStmtValue>
+ | <DataStmtValueList> -:T_COMMA <DataStmtValue>
+\end{verbatim}}
-\subsection{AST Structure}
+means that the AST will contain an object of type
+\texttt{List$<$ASTDataStmtValueNode$>$} whenever a \verb!<DataStmtValueList>!
+appears in the grammar. For instance, \# R533 (just described in the previous
+section) uses the \texttt{DataStmtValueList} construct. Notice in
+Listing~\ref{lst:astdatastmtsetnode_api} that the return type of
+\texttt{getDataStmtValueList} is a \texttt{List}.
-The Fortran AST comprises several hundred classes. The class names of AST nodes all begin with
-\texttt{AST}, and all are located in the \texttt{org.eclipse.photran.internal.core.parser} package.
-The AST classes are generated automatically from the parsing grammar,
-\texttt{fortran95.bnf}.
+Putting an object that implements the \texttt{java.util.List} into the tree
+(rather than having a chain of nodes) makes it easier to iterate through the
+list, determine its size and insert new child nodes.
-\subsubsection{Ordinary AST Nodes}
+\subsubsection{Annotation \#2: \texttt{(superclass)}}
-The children of any AST node are retrieved by calling one of its \texttt{get...}
-methods, which return \texttt{null} if that child is not present.
-For example, \texttt{fortran95.bnf} defines a \textit{$<$ProgramUnit$>$} as follows.
+The \texttt{(superclass)} annotation is used to create an interface that is
+implemented by all symbols on the right-hand side of the specification will
+implement. For example, the following specifications
{\footnotesize\begin{verbatim}
-# R202
-<ProgramUnit> ::=
- <MainProgram>
- | <FunctionSubprogram>
- | <SubroutineSubprogram>
- | <Module>
- | <BlockDataSubprogram>
+# R207
+(superclass):<DeclarationConstruct> ::=
+ | <DerivedTypeDef>
+ | <InterfaceBlock>
+ | <TypeDeclarationStmt>
+ | <SpecificationStmt>
+
+...
+
+# R214
+(superclass):<SpecificationStmt> ::=
+ | <AccessStmt>
+ | <AllocatableStmt>
+ | <CommonStmt>
+ | <DataStmt>
+ | <DimensionStmt>
+ | <EquivalenceStmt>
+ | <ExternalStmt>
+ | <IntentStmt>
+ | <IntrinsicStmt>
+ | <NamelistStmt>
+ | <OptionalStmt>
+ | <PointerStmt>
+ | <SaveStmt>
+ | <TargetStmt>
+ | <UnprocessedIncludeStmt>
\end{verbatim}}
-An \texttt{ASTProgramUnitNode} object, then, provides the following interface.
+mean that an \textbf{interface} -- not a class -- named
+\texttt{ISpecificationStmt} will be generated for \# R214, and
+\texttt{ASTAccessStmtNode}, \texttt{ASTAllocatableStmtNode},
+\texttt{ASTCommonStmtNode}, etc will implement that interface. In addition,
+because \verb!<SpecificationStmt>! is used inside \# R207 which also uses the
+\texttt{(superclass):} annotation, \texttt{ISpecificationStmt} also extends the
+\texttt{IDeclarationConstruct} interface from \# R207 i.e.
-\begin{lstlisting}
-public class ASTProgramUnitNode extends ParseTreeNode
-{
- public ASTMainProgramNode getASTMainProgram() { ... }
- public ASTFunctionSubprogramNode getASTFunctionSubprogram() { ... }
- public ASTSubroutineSubprogramNode getASTSubroutineSubprogram() { ... }
- public ASTModuleNode getASTModule() { ... }
- public ASTBlockDataSubprogramNode getASTBlockDataSubprogram() { ... }
-}
+\begin{lstlisting}[numbers=none]
+ public interface ISpecificationStmt extends IASTNode, IDeclarationConstruct
\end{lstlisting}
-\subsubsection{List Nodes (Recursive Productions)}
+So, it is possible for an AST node to implement multiple interfaces based on the
+specifications in the grammar.
-Recursive productions are treated specially, since they are used frequently to
-express lists in the grammar. The recursive member is labeled in the grammar
-with an @ symbol. For example,
+Using the \texttt{(superclass)} annotation gives those nonterminals in \# R214
+nodes a common type; most notably, a \texttt{Visitor} can override the
+\texttt{visit(ISpecificationStmt)} method to treat all three node types
+uniformly.
-{\footnotesize\begin{verbatim}
-<Body> ::=
- <BodyConstruct>
- | @:<Body> <BodyConstruct>
-\end{verbatim}}
+\subsubsection{Annotation \#3: \texttt{(bool)}}
-indicates that a \textit{$<$Body$>$} consists of several \textit{$<$BodyConstruct$>$}s.
-They can be iterated with code such as the following.
+The \texttt{(bool)} annotation indicates that an accessor method will return a
+\lstinline!boolean! rather than an actual AST node. For example, the following
+specification
{\footnotesize\begin{verbatim}
-for (int i = 0, size = astBodyNode.size(); i < size; i++)
- doSomething(astBodyNode.getASTBodyConstruct(i);
+# R511
+<AccessSpec> ::=
+ | isPublic(bool):T_PUBLIC
+ | isPrivate(bool):T_PRIVATE
\end{verbatim}}
-\subsubsection{Elevated Nodes}
+will generate the \texttt{ASTAccessSpecNode} class as shown in Listing~\ref{lst:astaccessspecnode_api}.
+
+\begin{code}
+\begin{lstlisting}
+public class ASTAccessSpecNode extends ASTNode
+{
+ // in ASTAccessSpecNode
+ Token isPrivate;
+ // in ASTAccessSpecNode
+ Token isPublic;
+
+ public boolean isPrivate() {...}
+
+ public void setIsPrivate(Token newValue) {...}
+
+ public boolean isPublic() {...}
+
+ public void setIsPublic(Token newValue) {...}
+
+ ...
+}
+\end{lstlisting}
+\caption{ASTAccessSpecNode generated from \# R511}
+\label{lst:astaccessspecnode_api}
+\end{code}
+Notice on lines 8 \& 12 that the methods return \lstinline!boolean! values
+instead of \texttt{ASTNode}s. The \lstinline!boolean! values are usually used to
+test the presence of that particular \texttt{ASTNode} in the source code.
+
+\subsubsection{Annotation \#4: Labels}
+
+Specification \# R511 also illustrates the use of \emph{labels} in the grammar
+file: \verb!isPublic(bool):T_PUBLIC! results in a method called
+\texttt{isPublic} instead of \verb!getT_PUBLIC!. The use of labels can greatly
+enhance the readability of the program by making its intent clearer.
+
+\subsubsection{Annotation \#5: \texttt{(inline)}}
-Consider the following productions for a main program in Fortran.
+Consider the following specifications for a main program in Fortran:
{\footnotesize\begin{verbatim}
+# R1101
<MainProgram> ::=
| <MainRange>
| <ProgramStmt> <MainRange>
<MainRange> ::=
- <Body> <EndProgramStmt>
+ | <Body> <EndProgramStmt>
| <BodyPlusInternals> <EndProgramStmt>
- | <EndProgramStmt>
+ |
\end{verbatim}}
-From the standpoint of a typical Fortran user,
-a main program consists of a Program statement, a body (list of statements), perhaps some internal
-subprograms, and an End Program statement.
-This does not match the definition of a MainProgram in the parsing grammar above: the body and End Program
-statement are relegated to a separate MainRange nonterminal.
+From the standpoint of a typical Fortran programmer, a main program consists of
+a Program statement, a body (list of statements), perhaps some internal
+subprograms, and an End Program statement. This does not match the definition of
+a \verb!<MainProgram>! in the parsing grammar above: \verb!<Body>! and
+\verb!<EndProgStmt>! are relegated to a separate \verb!<MainRange>! nonterminal.
-The solution is to label the MainRange nonterminal with a caret (\^), indicating that it is an ``elevated'' or
-``pulled-up'' node.
+The solution is to label the MainRange nonterminal with the \texttt{(inline)} annotation, indicating that it is to be in-lined:
{\footnotesize\begin{verbatim}
-<MainProgram> ::=
- | ^:<MainRange>
- | <ProgramStmt> ^:<MainRange>
+# R1101
+(customsuperclass=ScopingNode):<MainProgram> ::=
+ | (inline):<MainRange>
+ | <ProgramStmt> (inline):<MainRange>
<MainRange> ::=
- <Body> <EndProgramStmt>
- | <BodyPlusInternals> <EndProgramStmt>
- | <EndProgramStmt>
+ | <Body> <EndProgramStmt>
+ | (inline):<BodyPlusInternals> <EndProgramStmt>
+ |
\end{verbatim}}
-This means that accessor methods that would otherwise be in a separate MainRange object will be placed in the
-MainProgram object instead. This means that an \texttt{ASTMainProgramNode} object has the following interface.
+This means that accessor methods that would otherwise be in a separate
+\texttt{ASTMainRangeNode} class will be placed in the
+\texttt{ASTMainProgramNode} class instead.
+Listing~\ref{lst:astmainprogramnode_api} shows that the accessors that were
+previously in \texttt{ASTMainRangeNode} have been in-lined to
+\texttt{ASTMainProgramNode}. Now there is no longer any need for a
+\texttt{ASTMainRangeNode} class.
-\begin{lstlisting}[numbers=none]
- public ASTProgramStmtNode getProgramStmt();
- public ASTBodyNode getBody();
- public ASTBodyPlusInternalsNode getBodyPlusInternals();
- public ASTEndProgramStmtNode getEndProgramStmt();
+\begin{code}
+\begin{lstlisting}
+public class ASTMainProgramNode extends ScopingNode implements IProgramUnit
+{
+ public ASTProgramStmtNode getProgramStmt()
+
+ public void setProgramStmt(ASTProgramStmtNode newValue)
+
+ public IASTListNode<IBodyConstruct> getBody()
+
+ public void setBody(IASTListNode<IBodyConstruct> newValue)
+
+ public ASTContainsStmtNode getContainsStmt()
+
+ public void setContainsStmt(ASTContainsStmtNode newValue)
+
+ public IASTListNode<IInternalSubprogram> getInternalSubprograms()
+
+ public void setInternalSubprograms(IASTListNode<IInternalSubprogram> newValue)
+
+ public ASTEndProgramStmtNode getEndProgramStmt()
+
+ public void setEndProgramStmt(ASTEndProgramStmtNode newValue)
+
+ ...
+}
\end{lstlisting}
+\caption{\texttt{ASTMainProgramNode} generated from \# R1101}
+\label{lst:astmainprogramnode_api}
+\end{code}
+
+\subsubsection{Annotation \#6: \texttt{(customsuperclass=*)}}
+
+Specification \# R1101 in the previous section also illustrates the use of the
+\\ \texttt{(customsuperclass=ScopingNode)} annotation. This makes
+\texttt{ScopingNode} the parent of the \texttt{ASTMainProgramNode} class. Note
+that \texttt{ScopingNode} (or whichever custom super class is chosen) has to be
+a descendant of \texttt{ASTNode} because every node in the AST has to be of that
+type (either directly or as a descendant).
+
+The \texttt{(customsuperclass=*)} annotation is a useful technique for
+delegating external methods that cannot be expressed through the grammar BNF
+file into a separate hand-coded class while still having the benefits of an
+auto-generated parser and AST.
-\subsubsection{Tokens}
+\subsection{Tokens}
+\label{sub:tokens}
\texttt{Token}s form the leaves of the AST. They record, among other things,
+
\begin{itemize}
\item The terminal symbol in the grammar that the token is an instance of (\texttt{getTerminal()})
\item The actual text of the token (\texttt{getText()})
-\item The line, column, offset, and length of the token text in the source file (\texttt{getLine(), getCol(), getOffset(), getLength()})
+\item The line, column, offset, and length of the token text in the source file (\texttt{getLine(), getCol(), getFileOffset(), getLength()})
\end{itemize}
+
Most of the remaining fields are used internally for refactoring.
+\subsection{Fortran Program AST Example}
+
+The previous sections describe the conventions and additional annotations that
+are used to construct the AST nodes. While the conventions and annotations
+themselves are simple, the Fortran grammar is extremely complicated and contains
+hundreds of grammar specifications. Essentially, even the simplest Fortran
+program might contain a very complicated AST. For instance this simple Fortran
+program: \\
+
+\begin{lstlisting}[language=Fortran, frame=lines]
+program main
+ integer a
+end
+\end{lstlisting}
+
+generates the AST shown in Figure~\ref{fig:simple_fortran_ast}. As an exercise,
+the reader is encouraged to derive the structure of the AST from the grammar
+specifications in the \emph{fortran.bnf} file beginning with \# R201,
+\verb!<ExecutableProgram>!.
+
+Moreover, modifying our previous program slightly to contain an array as shown
+below, causes the AST structure to become extremely complicated. The reader is
+encouraged to create a file containing this program and view the resulting AST
+in the Outline View in Photran. Notice how the \verb!(5)! expression for
+specifying the dimension of the array requires building up an entire hierarchy
+of \texttt{ASTLevel5ExprNode}, \texttt{ASTLevel4ExprNode},
+\texttt{ASTLevel3ExprNode}, \ldots \\
+
+\begin{lstlisting}[language=Fortran, frame=lines]
+program main
+ integer a(5) ! This declares an array of 5 integers
+end
+\end{lstlisting}
+
+
+\begin{image}
+ \centering
+ \includegraphics[height=4in]{images/simple_fortran_AST.png}
+ \caption{AST for simple Fortran program as viewed through the Outline View}
+ \label{fig:simple_fortran_ast}
+\end{image}
+
+Fortunately, it is not necessary to know every specification in the grammar. For
+most refactoring and program analysis tasks, it is sufficient to rely on the
+information that the VPG provides. If that is insufficient, then it is usually
+enough to construct a Visitor to visit \emph{only} the nodes of interest and
+``collect'' the information that is required.
+
\section{Scope and Binding Analysis}
-Certain nodes in a Fortran AST represent a lexical scope. All of these are declared as subclasses of
-\texttt{ScopingNode.}
+Currently, the only semantic analysis performed by Photran is binding analysis:
+mapping \emph{identifiers} to their \emph{declarations}. Compilers usually do
+this using symbol tables but Photran uses a more IDE/refactoring-based approach.
+
+Certain nodes in a Fortran AST represent a lexical scope. All of these nodes are
+declared as subclasses of \texttt{ScopingNode}:
+
\begin{itemize}
\item ASTBlockDataSubprogramNode
\item ASTDerivedTypeDefNode
\item ASTExecutableProgramNode
\item ASTFunctionSubprogramNode
-\item ASTInterfaceBlockNode\footnote{An interface block defines a nested scope only if it is a named interface.
- Anonymous (unnamed) interfaces provide signatures for subprograms in their
- enclosing scope.}
+\item ASTInterfaceBlockNode\footnote{An interface block defines a nested scope only if it is a named interface.Anonymous (unnamed) interfaces provide signatures for subprograms in their enclosing scope.}
\item ASTMainProgramNode
\item ASTModuleNode
\item ASTSubroutineSubprogramNode
\end{itemize}
-The enclosing scope of a \texttt{Token} can be retrieved by calling the following method on the \texttt{Token} object.
+Each of the subclasses of \texttt{ScopingNode} represents a scoping unit in
+Fortran. The \texttt{ScopingNode} class has several public methods that provide
+information about a scope. For example, one can retrieve a list of all of the
+symbols declared in that scope; retrieve information about its
+\texttt{IMPLICIT} specification; find its header statement (e.g. a
+\texttt{FUNCTION} or \texttt{PROGRAM} statement); and so forth.
+
+The enclosing scope of a \texttt{Token} can be retrieved by calling the
+following method on the \texttt{Token} object:
-{\footnotesize\begin{verbatim}
- public ScopingNode getEnclosingScope()
-\end{verbatim}}
+\begin{lstlisting}[numbers=none]
+public ScopingNode getEnclosingScope()
+\end{lstlisting}
-Identifier tokens (\texttt{Token}s for which \texttt{token.getTerminal() == Terminal.T\_IDENT}),
-which represent functions, variables, etc. in the Fortran grammar, are \emph{bound} to a declaration.
-(The introduction to VPGs (URL above) provides an example visually.) Although, ideally, every identifier
-will be bound to exactly one declaration, this is not always the case (the programmer may have written incorrect code,
-or Photran may not have enough information to resolve the binding uniquely). So the \texttt{resolveBinding} method
-on a token has the following signature:
+Identifier tokens (\texttt{Token}s for which \texttt{token.getTerminal() ==
+Terminal.T\_IDENT}), which represent functions, variables, etc. in the Fortran
+grammar, are \emph{bound} to a declaration\footnote{The introduction to VPGs
+earlier in this chapter (URL above) provides an example visually.}. Although,
+ideally, every identifier will be bound to exactly one declaration, this is not
+always the case: the programmer may have written incorrect code, or Photran may
+not have enough information to resolve the binding uniquely). So the
+\texttt{resolveBinding} method returns a \emph{list} of \texttt{Definition}
+objects:
-{\footnotesize\begin{verbatim}
- public List<Definition> resolveBinding()
-\end{verbatim}}
+\begin{lstlisting}[numbers=none]
+public List<Definition> resolveBinding()
+\end{lstlisting}
+
+A \texttt{Definition} object contains many public methods which provide a wealth
+of information. From a \texttt{Definition} object, it is possible to get a list
+of all the references to a particular declaration (using
+\texttt{findAllReferences}) and where that particular declaration is located in
+the source code (using \texttt{getTokenRef}). Both of these methods return a
+\texttt{PhotranTokenRef} object. See Section~\ref{sub:token_or_tokenref} for a
+comparison between \texttt{Token} and \texttt{TokenRef}.
+
+\subsection{Examples of Binding Analysis}
+
+\subsubsection{Obtaining the \texttt{Definition} of a variable}
+
+If you have a reference to the \texttt{Token} object of that variable (for
+instance through iterating over all \texttt{Tokens} in the current Fortran AST)
+then use:
+
+\begin{lstlisting}[numbers=none, frame=lines]
+// myToken is the reference to that variable
+List<Definition> bindings = myToken.resolveBinding();
+
+if(bindings.size() == 0)
+ throw new Exception(myToken.getText() + " is not declared");
+else if (bindings.size() > 1)
+ throw new Exception(myToken.getText() + " is an ambiguous reference");
+
+Definition definition = bindings.get(0);
+\end{lstlisting}
-The \texttt{Definition} object contains information about what type of entity was defined (local variable, function,
-common block, etc.), array information (if applicable), whether it was defined implicitly, etc.
+If you do \textbf{not} have a reference to a \texttt{Token} but you know the
+name of the identifier, you can first construct a \emph{hypothetical}
+\texttt{Token} representing an identifier and search for that in a
+\emph{particular} \texttt{ScopingNode} (possibly obtained by calling the static
+method \texttt{ScopingNode.getEnclosingScope(IASTNode node)}).
+\\
-\section{Type Checking}
+\begin{lstlisting}[numbers=none, frame=lines]
+Token myToken = new Token(Terminal.T_IDENT, "myNameOfIdentifier");
+List<PhotranTokenRef> definitions = myScopingNode.manuallyResolve(myToken);
+\end{lstlisting}
+
+If you want to search for the identifier in \textbf{all} \texttt{ScopingNodes}
+for the current source file, then retrieve all the \texttt{ScopingNodes} and
+manually iterate through each one. Remember that the root of the AST is a
+\texttt{ScopingNode} and you may obtain the root of the AST through the
+\texttt{getRoot} method declared in \texttt{IFortranAST}.
+\\
+
+\begin{lstlisting}[numbers=none, frame=lines]
+List<ScopingNode> scopes = myRoot.getAllContainedScopes();
-The type of any expression (\texttt{ASTExprNode}) can be determined by invoking the static method
-\texttt{TypeChecker.getTypeOf(node)}. At the time of writing, this behavior is not completely implemented.
+for (ScopingNode scopingNode : scopes)
+{
+ // search through each ScopingNode
+}
+\end{lstlisting}
+\subsubsection{Examples in \texttt{FortranEditorASTActionDelegate} subclasses}
+The following subclasses of \texttt{FortranEditorASTActionDelegate} all contain short working examples of how to use the binding analysis API in Photran:
+\begin{itemize}
+ \item DisplaySymbolTable
+ \item FindAllDeclarationsInScope
+ \item OpenDeclaration
+ \item SelectEnclosingScope
+\end{itemize}
\section{How to Get Acquainted with the Program Representation}
+\label{sec:how_to_get_acquainted_with_the_program_representation}
+
+\emph{All these features work on \textbf{Fortran projects}. A new Fortran
+project can be created via File $>$ New $>$ Fortran $>$ Fortran Project. These
+features do not work on individual Fortran source files.}
\subsection{Visualizing ASTs}
-Photran can display ASTs rather than the ordinary Outline view.
-This behavior can be enabled from the Fortran workspace preferences
-(click on Window $>$ Preferences in Windows/Linux, or
-Eclipse $>$ Preferences in Mac OS X). Clicking on an AST node in the Outline
-view will move the cursor to that construct's position in the source file.
+Photran can display ASTs in place of the ordinary Outline view.
+This behavior can be enabled from the Fortran workspace preferences:
+
+\begin{itemize}
+ \item Click on Window $>$ Preferences in Windows/Linux, orEclipse $>$ Preferences in Mac OS X.
+ \item Select ``Fortran'' on the left hand side of the preference dialog (do not expand it).
+ \item Select ``(Debugging) Show entire parse tree rather than Outline view''
+\end{itemize}
+
+Clicking on an AST \texttt{node}Token in the Outline view will move the cursor
+to that construct's position in the source file.
\subsection{Visually Resolving Bindings}
-In a Fortran editor, click on an identifier (positioning the cursor over it),
-and press F3 (or click Navigate $>$ Open Declaration, or right-click and choose Open Declaration.) The
-binding will be resolved and the declaration highlighted. If there are multiple bindings, a popup window will open
-and one can be selected. If the identifier is bound to a declaration in a module defined in a different file, an
-editor will be opened on that file.
+This feature is disabled by default since it requires performing an update on
+the VPG database each time the file is saved -- an operation that could be
+expensive for larger files. To enable this feature, right-click on the current
+Fortran project folder (\textbf{not} the individual Fortran source file) in the
+Fortran Projects View and select ``Properties''. In the dialog that pops-up,
+navigate to Fortran General $>$ Analysis/Refactoring. Select the ``Enable
+Fortran Declaration view'' checkbox.
+
+Then, in a Fortran editor, click on an identifier (position the cursor over
+it), and press F3 (or click Navigate $>$ Open Declaration, or right-click and
+choose Open Declaration.) The binding will be resolved and the declaration
+highlighted. If there are multiple bindings, a pop-up window will open and one
+can be selected. If the identifier is bound to a declaration in a module defined
+in a different file, an editor will be opened on that file.
\subsection{Visualizing Enclosing Scopes}
-Click on any token in the Fortran editor, and click Refactor $>$ (Debugging) $>$ Select Enclosing Scope.
-The entire range of source text for that token's enclosing \texttt{ScopingNode} will be highlighted.
+Click on any token in the Fortran editor, and click Refactor $>$ (Debugging) $>$
+Select Enclosing Scope. The entire range of source text for that token's
+enclosing \texttt{ScopingNode} will be highlighted.
\subsection{Visualizing Definitions}
-Open a file in the Fortran editor, and
-click Refactor $>$ (Debugging) $>$ Display Symbol Table for Current File.
-Indentation shows scope nesting, and each line summarizes the information in a \texttt{Definition} object.
+Open a file in the Fortran editor, and click Refactor $>$ (Debugging) $>$
+Display Symbol Table for Current File. Indentation shows scope nesting, and each
+line summarizes the information in a \texttt{Definition} object.
diff --git a/org.eclipse.photran-dev-docs/dev-guide/plugins.ltx-inc b/org.eclipse.photran-dev-docs/dev-guide/plugins.ltx-inc
index 48022a52..c65d05d9 100644
--- a/org.eclipse.photran-dev-docs/dev-guide/plugins.ltx-inc
+++ b/org.eclipse.photran-dev-docs/dev-guide/plugins.ltx-inc
@@ -1,137 +1,155 @@
% Plug-in Decomposition
+\section{Introduction}
-\textit{Last Updated 4/4/07}
+This chapter presents a high-level overview of the different projects and
+plug-ins in Photran. It serves as a guide for developers reverse-engineering
+Photran to \emph{guess-and-locate} where certain components are. It also serves
+as a guide for contributers on \emph{where} to put their contributions.
-The following projects comprise the ``base'' of Photran.
-All sources in these projects are Java 4.
+The following sections are grouped by feature. A feature is a collection of
+related Eclipse plug-ins that the user can install as a whole.
+
+\section{Base Photran Feature: \\org.eclipse.photran-feature}
+
+The following projects comprise the ``base'' of Photran. All sources in these projects are Java 4.
+
+\begin{itemize}
+
+\item \textbf{org.eclipse.photran.cdtinterface}
+
+This contains all of the components (core and user interface) related
+to integration with the CDT. It includes:
+\begin{itemize}
+\item FortranLanguage for Fortran (i.e., the means of adding Fortran to the list
+of languages recognized by the CDT)
+\item Fortran model elements, and icons for the Outline and Projects Views
+\item \emph{Simple} lexical analyzer-based model builder and an extension
+point for contributing more sophisticated model builders
+\item Fortran perspective, Fortran Projects view, and new project wizards
+\end{itemize}
+
+For more information about CDT, see Chapter~\ref{cha:cdt}.
+
+\item \textbf{org.eclipse.photran.core}
+
+This is the Photran Core plug-in. It contains much of the Fortran-specific
+``behind the scenes'' functionality:
+
+\begin{itemize}
+\item Utility classes
+\item Error parsers for Fortran compilers
+\item Fortran 95 lexical analyzer
+\item Workspace preferences
+\end{itemize}
+
+\item \textbf{org.eclipse.photran.managedbuilder.core}, \\
+ \textbf{org.eclipse.photran.managedbuilder.gnu.ui}, \\
+ \textbf{org.eclipse.photran.managedbuilder.ui}
+
+Support for Managed Build projects using the GNU toolchain. Managed by Craig Rasmussen at LANL.
+
+\item \textbf{org.eclipse.photran.ui}
+
+This contains the Fortran-specific components of the user interface:
\begin{itemize}
+\item Fortran Editors
+\item Preference pages
+\end{itemize}
-\item org.eclipse.photran-feature
+\end{itemize}
- This is the Eclipse feature for Photran, used to build the Zip
- file distributed to users. (A feature is a grouping of related
- plug-ins.)
+\section{Virtual Program Graph (VPG) feature: \\org.eclipse.photran.vpg-feature}
-\item org.eclipse.photran.cdtinterface
+The following projects support parsing, analysis, and refactoring of Fortran
+sources. They are written in Java 5. The Virtual Program Graph is described in
+more detail in Chapter~\ref{cha:parsing}.
- This contains all of the components (core and user interface) related
- to integration with the CDT. It includes
- \begin{itemize}
- \item ILanguage for Fortran (i.e., the means
- of adding Fortran to the list of languages recognized
- by the CDT)
- \item Fortran model elements, and icons for the Outline and Projects views
- \item Simple lexical analyzer-based model builder, and an extension
- point for contributing more sophisticated model builders
- \item Fortran perspective, Fortran Projects view, and new project wizards
- \end{itemize}
+\begin{itemize}
-\item org.eclipse.photran.core
+\item \textbf{org.eclipse.photran.vpg.core}
- This is the Photran Core plug-in. It contains much of the
- Fortran-specific ``behind the scenes'' functionality:
- \begin{itemize}
- \item Utility classes
- \item Error parsers for Fortran compilers
- \item Fortran 95 lexical analyzer
- \item Workspace preferences
- \end{itemize}
+This contains the parsing, analysis, and refactoring infrastructure.
+\begin{itemize}
+\item Fortran parser and abstract syntax tree (AST)
+\item Fortran preprocessor (to handle INCLUDE lines)
+\item Parser-based model builder
+\item Virtual Program Graph library (vpg-eclipse.jar)
+\item Photran's Virtual Program Graph (VPG)
+\item Utility classes (e.g., \texttt{SemanticError}, \texttt{LineCol})
+\item Project property pages
+\item Binding analysis (equivalent to symbol tables)
+\item Refactoring/program transformation engine
+\item Refactorings
+\end{itemize}
-\item org.eclipse.photran.core.errorparsers.xlf
+\item \textbf{org.eclipse.photran.core.vpg.tests}, \\
+ \textbf{org.eclipse.photran.core.vpg.tests.failing}
- Error parser for the XLF compiler.
- Managed by Craig Rasmussen at LANL.
+JUnit Plug-in tests for the VPG core plug-in.
-\item org.eclipse.photran.managedbuilder.core, \\
- org.eclipse.photran.managedbuilder.gnu.ui, \\
- org.eclipse.photran.managedbuilder.xlf.ui, \\
- org.eclipse.photran.managedbuilder.ui
+All tests in org.eclipse.photran.core.vpg.tests should pass. Test suites and
+test cases are placed in the ``failing'' plug-in project until all of their
+tests pass.
- Support for Managed Build projects using the GNU toolchain.
- Managed by Craig Rasmussen at LANL.
+These plug-ins \textit{must} be run as a ``JUnit Plug-in Test'' ()\textbf{not} a
+``JUnit Test''). In the launch configuration, the environment variable TESTING
+must be set to some non-empty value. (This ensures that the VPG will not try to
+run in the background and interfere with testing.)
-\item org.eclipse.photran.core.intel, \\
- org.eclipse.photran.intel-feature, \\
- org.eclipse.photran.managedbuilder.intel.ui
+\item \textbf{org.eclipse.photran.ui.vpg}
- Support for Managed Build projects using Intel toolchains.
- Maintained by Bill Hilliard at Intel.
+UI contributions that depend on the \texttt{org.eclipse.photran.core.vpg}
+plug-in. Currently, this includes (1) the Open Declaration action, a project
+property page where the user can customize the search path for Fortran modules
+and include files, and (2) all of the actions in the Refactoring menu. Search
+and other VPG-dependent features will be placed here in the future.
-\item org.eclipse.photran.ui
+\end{itemize}
+
+\section{XL Fortran Compiler Feature: \\org.eclipse.photran.xlf-feature}
- This contains the Fortran-specific components of the user interface:
- \begin{itemize}
- \item Editor
- \item Preference pages
- \end{itemize}
+The following are plug-ins to support the
+\href{http://www-306.ibm.com/software/awdtools/fortran/}{XL Fortran compiler}.
+
+\begin{itemize}
+\item \textbf{org.eclipse.photran.core.errorparsers.xlf}, \\
+ \textbf{org.eclipse.photran.managedbuilder.xlf.ui}
+
+Support for Managed Build projects using XL toolchains. Managed by Craig
+Rasmussen at LANL.
\end{itemize}
-\noindent
-The following projects support parsing, analysis, and refactoring of Fortran sources.
-They are written in Java 5.
-The Virtual Program Graph is described in more detail later.
+\section{Intel Fortran Compiler Feature: \\org.eclipse.photran.intel-feature}
+
+The following are plug-ins to support the \href{http://www.intel.com/cd/software/products/asmo-na/eng/compilers/284132.htm}{Intel Fortran Compiler}.
\begin{itemize}
+\item \textbf{org.eclipse.photran.core.intel}, \\
+ \textbf{org.eclipse.photran.managedbuilder.intel.ui}
-\item org.eclipse.photran.vpg.core
-
- This contains the parsing, analysis, and refactoring infrastructure.
- \begin{itemize}
- \item Fortran parser and abstract syntax tree (AST)
- \item Fortran preprocessor (to handle INCLUDE lines)
- \item Parser-based model builder
- \item Virtual Program Graph library (vpg-eclipse.jar)
- \item Photran's Virtual Program Graph (VPG)
- \item Utility classes (e.g., \texttt{SemanticError}, \texttt{LineCol})
- \item Project property pages
- \item Binding analysis (equivalent to symbol tables)
- \item Type checker (incomplete)
- \item Refactoring/program transformation engine
- \item Refactorings
- \end{itemize}
-
-\item org.eclipse.photran.core.vpg.tests, \\
- org.eclipse.photran.core.vpg.tests.failing
-
- JUnit Plug-in tests for the VPG core plug-in.
-
- All tests in org.eclipse.photran.core.vpg.tests should pass.
- Test suites and test cases are placed in the ``failing'' plug-in until all of their tests pass.
-
- These plug-ins \textit{must} be run as a ``JUnit Plug-in Test,'' not a ``JUnit Test.''
- In the launch configuration, the environment variable TESTING must be set to some non-empty value.
- (This ensures that the VPG will not try to run in the background and interfere with testing.)
-
-\item org.eclipse.photran.ui.vpg
-
- UI contributions that depend on the org.eclipse.photran.core.vpg plug-in.
- Currently, this is the
- Open Declaration action,
- a project property page where the user can customize the search path for Fortran modules and include files,
- and all of the actions in the Refactoring menu.
- Search and other VPG-dependent features will be placed here in the future.
+Support for Managed Build projects using Intel toolchains. Maintained by Bill
+Hilliard at Intel.
\end{itemize}
-\noindent
+\section{Non-plug-in projects}
+
The following projects are in CVS but are not distributed to users:
\begin{itemize}
-\item org.eclipse.photran-dev-docs
+\item \textbf{org.eclipse.photran-dev-docs}
- Developer documentation, including this document
- (\texttt{dev-guide/*}), CVS instructions
- (\texttt{dev-guide/cvs-instructions.pdf}),
- the materials from our presentation at EclipseCon 2006 on adding a new
- language to the CDT, and a spreadsheet
- mapping features in the Fortran language to JUnit tests
- (\texttt{language-coverage/*}).
+Developer documentation, including this document (\texttt{dev-guide/*}), CVS
+instructions (\texttt{dev-guide/cvs-instructions.pdf}), the materials from our
+presentation at EclipseCon 2006 on adding a new language to the CDT, and a
+spreadsheet mapping features in the Fortran language to JUnit tests
+(\texttt{language-coverage/*}).
-\item org.eclipse.photran-samples
+\item \textbf{org.eclipse.photran-samples}
- A Photran project containing an assortment of Fortran code.
+A Photran project containing an assortment of Fortran code.
\end{itemize}
diff --git a/org.eclipse.photran-dev-docs/dev-guide/refactoring.ltx-inc b/org.eclipse.photran-dev-docs/dev-guide/refactoring.ltx-inc
index 3321c0e5..74255477 100644
--- a/org.eclipse.photran-dev-docs/dev-guide/refactoring.ltx-inc
+++ b/org.eclipse.photran-dev-docs/dev-guide/refactoring.ltx-inc
@@ -1,81 +1,333 @@
% Refactoring
-\textit{Last Updated 10/2/07}
+\section{Introduction}
-\section{Structure of a Refactoring}
+A refactoring is a program transformation to improve the quality of the source
+code by making it easier to understand and modify. A refactoring is a special
+kind of transformation because it preserves the \emph{observable behavior} of
+your program -- it neither removes nor adds any functionality.\footnote{For more
+information see
+\href{http://www.amazon.com/Refactoring-Improving-Existing-Addison-Wesley-Technology/dp/0201485672}{Refactoring:
+Improving the Design of Existing Code}}.
-Refactorings in Photran are subclassed from \texttt{FortranRefactoring},
-which is in turn a subclass of the \texttt{Refactoring} class provided
-by the Eclipse Language ToolKit (LTK).
+As mentioned in Chapter~\ref{cha:introduction}, the purpose in writing
+Photran was to create a refactoring tool for Fortran. Because Photran is
+structured as a plug-in for Eclipse, we can take advantage and reuse many of the
+language-neutral support that Eclipse provides for refactoring. This makes it
+possible to create refactoring tools that \emph{resemble} the Java Development
+Tools that most Eclipse programmers are already familiar with.
-Refactorings must
-implement three methods:
+However, implementing first-class support for Fortran refactoring is not an easy
+task. It requires having an accurate representation of the underlying Fortran
+source files so that our tools can perform proper program analysis to construct
+our automated refactoring. The VPG (see Chapter~\ref{cha:parsing}) is our
+initial step in providing such a representation; the VPG will be improved in
+future versions of Photran to provide support for many different types of
+refactoring and program analysis.
-{\footnotesize\begin{verbatim}
- public String getName();
- protected abstract void doCheckInitialConditions(RefactoringStatus status, IProgressMonitor pm) throws PreconditionFailure;
- protected abstract void doCheckFinalConditions(RefactoringStatus status, IProgressMonitor pm) throws PreconditionFailure;
- protected abstract void doCreateChange(IProgressMonitor pm) throws CoreException, OperationCanceledException;
-\end{verbatim}}
+In this chapter, we describe how to add automated refactorings for Fortran using
+the underlying infrastructure provided by Eclipse (and Photran) as well as the
+analysis tools provided by the VPG.
-\texttt{getName} simply returns the name of the refactoring: ``Rename,'' ``Extract Subroutine,''
-``Introduce Implicit None,'' or something similar.
+\section{Structure of a Fortran Refactoring}
+\label{sec:structure_of_a_fortran_refactoring}
-Initial conditions are checked before any dialog is displayed to the user.
-An example would be making sure that the user has selected an identifier to rename.
-If the check fails, a \texttt{PreconditionFailure} should be thrown with a message
-describing the problem for the user.
+Refactorings in Photran are subclassed from \texttt{FortranRefactoring}, which
+is in turn a subclass of the \texttt{Refactoring} class provided by the Eclipse
+Language ToolKit (LTK)\footnote{See
+\href{http://www.eclipse.org/articles/Article-LTK/ltk.html}{The Language
+Toolkit: An API for Automated Refactorings in Eclipse-based IDEs} for an
+introduction to the LTK.}.
-Final conditions are checked after the user has provided any input.
-An example would be making sure that a string is a legal identifier.
+The LTK is a language-neutral API for supporting refactorings in the Eclipse
+environment. It provides a generic framework to support the following
+functionalities:
+\begin{enumerate}
+ \item Invoking the refactoring from the user interface (UI).
+ \item Presenting the user with a wizard to step through the refactoring.
+ \item Presenting the user with a preview of the changes to be made.
+\end{enumerate}
+
+In other words, the LTK provides a common UI for refactorings: This allows
+refactorings for Java, C/C++, and Fortran to all have the same look and feel.
+
+A concrete Fortran refactoring must implement the following \emph{four} methods:
+\\
+\begin{code}
+\begin{lstlisting}[numbers=none]
+public abstract String getName();
+
+protected abstract void doCheckInitialConditions(RefactoringStatus status,
+ IProgressMonitor pm) throws PreconditionFailure;
+
+protected abstract void doCheckFinalConditions(RefactoringStatus status,
+ IProgressMonitor pm) throws PreconditionFailure;
+
+protected abstract void doCreateChange(IProgressMonitor pm) throws
+ CoreException, OperationCanceledException;
+\end{lstlisting}
+\caption{Abstract methods of \texttt{FortranRefactoring} class}
+\label{lst:FortranRefactoring_API}
+\end{code}
+
+\texttt{getName} simply returns the name of the refactoring: ``Rename,''
+``Extract Subroutine,'' ``Introduce Implicit None,'' or something similar. This
+name will be used in the title of the wizard that is displayed to the user.
+
+Initial conditions are checked before any wizard is displayed to the user. An
+example would be making sure that the user has selected an identifier to rename.
+If the check fails, a \texttt{PreconditionFailure} should be thrown with a
+message describing the problem for the user.
+
+Final conditions are checked after the user has provided any input. An example
+would be making sure that the new name that that user has provided is a legal
+identifier.
+
+The actual transformation is done in the \texttt{doCreateChange} method, which
+will be called only after the final preconditions are checked. For more information, see Section~\ref{sec:ast_rewriting}.
+
+The \texttt{FortranRefactoring} class provides a large number of
+\texttt{protected} utility methods common among refactorings, such as a method
+to determine if a token is a uniquely-bound identifier, a method to parse
+fragments of code that are not complete programs, and a \texttt{fail} method
+which is simply shorthand for throwing a \texttt{PreconditionFailure}. It is
+worth reading through the source code for \texttt{FortranRefactoring} before
+writing your own utility methods.
+
+\section{Creating Changes: AST Rewriting}
+\label{sec:ast_rewriting}
+
+After determining the files that are affected and the actual changes that are
+required for a particular refactoring, manipulating the source code in the
+\texttt{doCreateChange} method is conceptually straightforward.
+
+Instead of manipulating the text in the files directly (by doing a textual find
+\& replace) we use a more scalable approach: manipulating the Abstract Syntax
+Tree (AST) of the source code. This allows us to make changes based on the
+program's semantics and its syntactic structure. This section assumes some
+familiarity with the AST used in Photran. For more information about the AST,
+refer to Section~\ref{sec:virtual_program_graph}.
+
+\subsection{Common Methods for Manipulating the AST}
+
+In the following paragraphs, we describe some of the approaches that are
+currently being used in Photran for manipulating the AST.
+
+\subsubsection{Changing the Text of \texttt{Token}s}
+
+To change the text of a single token, simply call its \texttt{setText} method.
+This is used in \texttt{RenameRefactoring} to rename tokens while preserving the
+``shape'' of the AST.
+\\
+\begin{code}
+\begin{lstlisting}[firstnumber=273, emph={setText}]
+private void makeChangesTo(IFile file, IProgressMonitor pm) throws Error
+{
+ try
+ {
+ vpg.acquirePermanentAST(file);
+
+ if (definitionToRename.getTokenRef().getFile().equals(file))
+ definitionToRename.getTokenRef().findToken().setText(newName);
+
+ for (PhotranTokenRef ref : allReferences)
+ if (ref.getFile().equals(file))
+ ref.findToken().setText(newName);
+
+ addChangeFromModifiedAST(file, pm);
+
+ vpg.releaseAST(file);
+ }
+ catch (Exception e)
+ {
+ throw new Error(e);
+ }
+}
+\end{lstlisting}
+\caption{Use of \texttt{setText} in \texttt{RenamingRefactoring} (see
+RenameRefactoring.java)}
+\end{code}
+
+\subsubsection{Removing/replacing AST Nodes}
+\label{ssub:removing_replacing_AST}
+
+To remove or replace part of an AST, call \texttt{replaceChild},
+\texttt{removeFromTree} or \texttt{replaceWith} on the node itself. These
+methods are defined in the \texttt{IASTNode} interface that all nodes implement.
+Line 107 of Listing~\ref{lst:introimplicitnone_doChange} shows an example of the
+\texttt{removeFromTree} method.
+\\
+
+\begin{code}
+\begin{lstlisting}[firstnumber=7236]
+public static interface IASTNode
+{
+ void replaceChild(IASTNode node, IASTNode withNode);
+ void removeFromTree();
+ void replaceWith(IASTNode newNode);
+ ...
+}
+\end{lstlisting}
+\caption{AST manipulation methods in \texttt{IASTNode} (see Parser.java) that
+all AST nodes implement}
+\end{code}
-The actual transformation is done in the \texttt{doCreateChange} method,
-which will be called only after the final preconditions are checked.
+In addition, if the \emph{specific} type of the AST is known, then it is
+possible to just call its \emph{setter} method to directly replace particular
+nodes. For more information on the available setters for each node type, see
+Section~\ref{sub:ordinary_ast_nodes}.
-The \texttt{FortranRefactoring} class provides a large number of \texttt{protected} utility methods common among
-refactorings, such as a method to determine if a token is a uniquely-bound identifier, a method to parse fragments
-of code that are not complete programs, and a \texttt{fail} method which is simply shorthand for throwing a
-\texttt{PreconditionFailure}.
+\subsubsection{Inserting new AST Nodes}
-\section{AST Rewriting}
+Some refactorings require inserting new AST nodes into the current program. For
+instance, the ``Intro Implicit None Refactoring'' inserts new declaration
+statements to make the type of each variable more explicit.
-After it is determined what files are affected by a refactoring, manipulating the source code
-in the \texttt{doCreateChange} method is conceptually straightforward.
+There are \emph{three} steps involved in inserting a new AST node:
\begin{enumerate}
-\item To change the text of a single token, simply call its \texttt{setText} method.
-\item To remove part of an AST, call the static method \texttt{SourceEditor.cut(node)}, which will remove
- the given node (and all of its children), returning it.
-\item To insert a node into an ast, call one of the \texttt{SourceEditor.paste...} methods. The pasted node
- will be automatically reindented to match its surroundings.
-\item To insert new nodes (e.g., hard-coded statements) into an AST, call one of the \texttt{parseLiteral...}
- methods inherited from the \texttt{FortranRefactoring} class to construct an AST fragment for the node, and
- then use \texttt{SourceEditor.paste...} to paste it at the appropriate position in the AST.
+ \item Constructing the new AST node.
+ \item Inserting the new AST node into the correct place.
+ \item Re-indenting the new AST node to fit within the current file.
\end{enumerate}
+\paragraph{Constructing the new AST node} The \texttt{FortranRefactoring} class
+provides convenience methods for constructing new AST nodes. These methods
+should be treated as part of the API for Fortran refactorings . For instance,
+the \texttt{parseLiteralStatement} methods constructs a list of AST nodes for
+use in the ``Intro Implicit None'' refactoring.
+
+\paragraph{Inserting the new AST node} Inserting the new AST node can be
+accomplished using the approach discussed previously in \emph{Removing/replacing AST Nodes}.
+
+\paragraph{Re-indenting the new AST node} It might be necessary to re-indent the
+newly inserted AST node so that it conforms with the indentation at its
+insertion point. The \texttt{Reindenter} utility class provides the static
+method \texttt{reindent} to perform this task. Refer to line 111 of
+Listing~\ref{lst:introimplicitnone_doChange}.
+\\
+
+\begin{code}
+\begin{lstlisting}[firstnumber=95,
+emph={removeFromTree, addChangeFromModifiedAST, Reindenter}]
+protected void doCreateChange(IProgressMonitor progressMonitor) throws
+CoreException, OperationCanceledException
+{
+ assert this.selectedScope != null;
+
+ for (ScopingNode scope : selectedScope.getAllContainedScopes())
+ {
+ if (!scope.isImplicitNone()
+ && !(scope instanceof ASTExecutableProgramNode)
+ && !(scope instanceof ASTDerivedTypeDefNode))
+ {
+ ASTImplicitStmtNode implicitStmt = findExistingImplicitStatement(scope);
+ if (implicitStmt != null) implicitStmt.removeFromTree();
+
+ IASTListNode<IBodyConstruct> newDeclarations = constructDeclarations(scope);
+ scope.getBody().addAll(0, newDeclarations);
+ Reindenter.reindent(newDeclarations, astOfFileInEditor);
+ }
+ }
+
+ this.addChangeFromModifiedAST(this.fileInEditor, progressMonitor);
+ vpg.releaseAllASTs();
+}
+\end{lstlisting}
+\caption{Inserting new declarations into an existing scope
+(see \texttt{IntroImplicitNoneRefactoring.java})}
+\label{lst:introimplicitnone_doChange}
+\end{code}
+
+\subsection{Committing Changes}
+
After all of the changes have been made to a file's AST,
-\texttt{addChangeFromModifiedAST(IFile file)} should be invoked to
-commit the change, after which it is safe to call \texttt{FortranWorkspace.getInstance().release...}.
-
-\textbf{CAUTION:} Internally, the AST is changed only enough to reproduce correct source code. After making changes
-to an AST, most of the accessor methods on \texttt{Token}s (\texttt{getLine(), getOffset(),} etc.)
-will return \textit{incorrect} values. This also means that the \texttt{find...} methods in \texttt{IFortranAST}
-will not work, and binding information will be incorrect. Due to the complex internal structure of the AST, accessor
-methods on AST nodes cannot be called either. Therefore, \textit{all program analysis should be done first;}
-pointers to all relevant nodes and tokens should be obtained \textit{prior to} making any modifications to the AST.
-In other words, it is best to consider the AST ``write-only'' as soon as any change has been made.
-
-\section{User Interface}
-
-Adding a refactoring to the user interface is best done by following an example.
-First, an action must be added to both the editor popup menu \textit{and} the Refactor
-menu in the menu bar by modifying the plugin.xml file in the org.eclipse.photran.refactoring.ui plug-in.
-Then, the action delegate must be created to populate the user interface of the refactoring wizard dialog;
-\texttt{RenameAction} (in the \texttt{org.eclipse.photran.internal.refactoring.ui} package)
-can be used as a starting point.
-
-\section{Examples: Rename and Introduce Implicit None}
-
-The Rename refactoring (\texttt{org.eclipse.photran.internal.core.refactoring.RenameRefactoring})
-and the Introduce Implicit None refactoring
-(\texttt{org.eclipse.photran.internal.core.refactoring.IntroImplicitNoneRefactoring})
-are non-trivial but readable and should serve as a model for building future Fortran refactorings.
+\texttt{addChangeFromModifiedAST} has to be invoked to actually
+commit the changes. This convenience function creates a new
+\texttt{TextFileChange} for the \emph{entire} content of the file. The
+underlying Eclipse infrastructure performs a \texttt{diff} internally to
+determine what parts have actually changed and present those changes to the user
+in the preview dialog.
+
+\section{Caveats}
+\label{sec:refactoring_caveats}
+
+\textbf{CAUTION:} Internally, the AST is changed only enough to reproduce
+correct source code. After making changes to an AST, most of the accessor
+methods on \texttt{Token}s (\texttt{getLine(), getOffset(),} etc.) will return
+\textit{incorrect} or \emph{null} values.
+
+Therefore, \textit{all program analysis should be done first}; pointers to all
+relevant \textbf{tokens} should be obtained (usually as \texttt{TokenRef}s)
+\textit{prior} to making any modifications to the AST. In general, ensure that
+all analysis (and storing of important information from \texttt{Token}s) should
+be done in the \texttt{doCheckInitialConditions} and
+\texttt{doCheckFinalConditions} methods of your refactoring before the
+\texttt{doCreateChange} method.
+
+\vspace{-0.2in}
+\subsection{\texttt{Token} or \texttt{TokenRef}?}
+\label{sub:token_or_tokenref}
+
+\texttt{Token}s form the leaves of the AST -- therefore they exist as part of
+the Fortran AST. Essentially this means that holding on to a reference to a
+\texttt{Token} object requires the entire AST to be present in memory.
+
+\texttt{TokenRef}s are lightweight descriptions of tokens in an AST. They
+contain only three fields: filename, offset and length. These three fields
+uniquely identify a particular token in a file. Because they are not part of the
+AST, storing a \texttt{TokenRef} does not require the entire AST to be present
+in memory.
+
+For most refactorings, using either \texttt{Token}s or \texttt{TokenRef}s does
+not make much of a difference. However, in a refactoring like ``Rename
+Refactoring'' that could potentially modify hundreds of files, it is impractical
+to store all ASTs in memory at once. Because of the complexity of the Fortran
+language itself, its ASTs can be rather large and complex. Therefore storing
+references to \texttt{TokenRef}s would minimize the number of ASTs that must be
+in memory.
+
+To retrieve an actual \texttt{Token} from a \texttt{TokenRef}, call the
+\texttt{findToken()} method in \texttt{PhotranTokenRef}, a subclass
+of \texttt{TokenRef}.
+
+To create a \texttt{TokenRef} from an actual \texttt{Token}, call the \texttt{getTokenRef} method in \texttt{Token}.
+
+\vspace{-0.2in}
+\section{Examples}
+
+The ``Rename'', ``Introduce Implicit None'' and ``Move COMMON To Module''
+refactorings found in the \texttt{org.eclipse.photran.internal.core.refactoring}
+package inside the \texttt{org.eclipse.photran.core.vpg} project are
+non-trivial but readable and should serve as a model for building future Fortran
+refactorings.
+
+An example of a simpler but rather \emph{useless} refactoring is presented in
+Appendix~\ref{app:obfuscate_refactoring}. It should be taken as a guide on the
+actual steps that are involved in registering a new refactoring with the UI and
+also how to actually construct a working Fortran refactoring.
+
+\section{Common Tasks}
+
+In this section, we briefly summarize some of the common tasks involved in
+writing a new Fortran refactoring.
+
+\noindent \textbf{In an AST, how do I find an ancestor node that is of a
+particular type?}
+\\ Sometimes it might be necessary to traverse the AST \emph{upwards} to look
+for an ancestor node of a particular type. Instead of traversing the AST
+manually, you should call the \texttt{findNearestAncestor(TargetASTNode.class)}
+method on a \texttt{Token} and pass it the \textbf{class} of the ASTNode that
+you are looking for.
+
+\noindent \textbf{How would I create a new AST node from a string?}
+\\ Call the \texttt{parseLiteralStatement(String string)} or
+\texttt{parseLiteralStatementSequence(String string)} method in
+\texttt{FortranRefactoring}. The former takes a \texttt{String} that represents
+a single statement while the latter takes a \texttt{String} that represents a
+sequence of statements.
+
+\noindent \textbf{How do I print the text of an AST node and all its children
+nodes?}
+\\ Call the \texttt{SourcePrinter.getSourceCodeFromASTNode(IASTNode node)}
+method. This method returns a \texttt{String} representing the source code of
+its parameter; it includes the user's comments, capitalization and whitespace.

Back to the top