Merged in hxcan/recoll (pull request #1): get active indexes from the environment

This commit is contained in:
medoc 2012-04-09 19:14:49 +02:00
parent 5bd071c5a6
commit 311decd6e6
68 changed files with 1474 additions and 624 deletions

View file

@ -9,15 +9,16 @@ Build-Depends: debhelper (>= 7),
libxapian-dev (>= 1.0.15),
libx11-dev,
libz-dev,
python-dev,
python-all-dev (>= 2.6.6-3~),
quilt
Standards-Version: 3.8.3
Standards-Version: 3.9.3
Package: recoll
Architecture: any
Depends: ${shlibs:Depends}, ${misc:Depends}
Recommends: aspell, python, xsltproc
Suggests: antiword, catdoc, ghostscript, libimage-exiftool-perl, poppler-utils, unrtf, python-mutagen
Recommends: aspell, python, xdg-utils, xsltproc
Suggests: antiword, catdoc, ghostscript, libimage-exiftool-perl, poppler-utils,
pstotext, python-chm, python-mutagen, unrtf, untex
Description: a personal full text search package with a QT GUI
The Recoll personal full text search package is based on a very strong
backend (Xapian), for which it provides an easy to use and feature-rich

View file

@ -1,113 +1,219 @@
This package was debianized by Jean-Francois Dockes <jfd@recoll.org> on
Wed, 10 Jan 2007 16:04:13 +0100.
Format: http://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
Upstream-Name: recoll
Upstream-Contact: Jean-Francois Dockes <jean-francois.dockes@wanadoo.fr>
Source: http://www.lesbonscomptes.com/recoll/
It was downloaded from http://www.recoll.org
Files: *
Copyright: 2005-2012, Jean-Francois Dockes <jean-francois.dockes@wanadoo.fr>
License: GPL-2+
Upstream Author: Jean-Francois Dockes <jfd@recoll.org>
Files: Binc IMAP project (bincimapmime/*)
Copyright: 2002-2005, Andreas Aardal Hanssen <andreas-binc@bincimap.org>
License: GPL-2+
Copyright: (C) 2005,2006, Jean-Francois Dockes <jfd@recoll.org>
Files: filters/rcl* files index/csguess.cpp internfile/htmlparse.cpp
Copyright: 2000-2004, Mikio Hirabayashi
License: GPL-2+
Files: filters/rclpython
Copyright: J\xfcrgen Hermann, Mike Brown, Christopher Arndt
<http://chrisarndt.de/en/software/python/colorize.html>
License: GPL-2+
Files: internfile/htmlparse.cpp mh_html.cpp
Copyright: 1999-2001, BrightStation PLC
Copyright: 2001, Ananova Ltd
Copyright: 2002-2004, Olly Betts
License: GPL-2+
Files: unac/*
Copyright: 2000-2002, Loic Dachary <loic@senga.org>
License: GPL-2+
Files: common/*
Copyright: 2004-2005, J.F.Dockes
License: GPL-2+
Files: debian/*
Copyright: 2007-2012, Kartik Mistry <kartik@debian.org>
License: GPL-2+
License: GPL-2+
This package is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
Foundation; either version 2 of the License, or (at your option) any later
version.
.
This package is distributed in the hope that it will be useful, but WITHOUT
ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
.
You should have received a copy of the GNU General Public License along with
this package; if not, write to the Free Software Foundation, Inc., 51 Franklin
St, Fifth Floor, Boston, MA 02110-1301 USA
.
On Debian systems, the complete text of the GNU General Public License can be
found in `/usr/share/common-licenses/GPL-2' and
`/usr/share/common-licenses/GPL-3'.
Files: aspell/*
Copyright: 2001-2002, by Kevin Atkinson
License: LGPL-2+
Files: images/text-x-python.png
Copyright: David Vignoni <http://www.oxygen-icons.org/>
License: LGPL-2+
License: LGPL-2+
This package is free software; you can redistribute it and/or modify it under
the terms of the GNU Lesser General Public License as published by the Free
Software Foundation; either version 2 of the License, or (at your option) any
later version.
.
This package is distributed in the hope that it will be useful, but WITHOUT
ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more
details.
.
You should have received a copy of the GNU Lesser General Public License along
with this package; if not, write to the Free Software Foundation, Inc., 51
Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
.
On Debian systems, the complete text of the GNU Lesser General Public License
can be found in `/usr/share/common-licenses/LGPL-2' and
`/usr/share/common-licenses/LGPL-2.1' and `/usr/share/common-licenses/LGPL-3'.
Files: qtgui/q3richtext_p.h
Copyright: 1992-2007, Trolltech ASA. All rights reserved
License:
This file is part of the Qt3Support module of the Qt Toolkit.
.
This file may be used under the terms of the GNU General Public License
version 2.0 as published by the Free Software Foundation and appearing in the
file LICENSE.GPL included in the packaging of this file. Please review the
following information to ensure GNU General Public Licensing requirements will
be met: http://trolltech.com/products/qt/licenses/licensing/opensource/
.
If you are unsure which license is appropriate for your use, please review the
following information:
http://trolltech.com/products/qt/licenses/licensing/licensingoverview
or contact the sales department at sales@trolltech.com.
.
In addition, as a special exception, Trolltech gives you certain additional
rights. These rights are described in the Trolltech GPL Exception version 1.0,
which can be found at http://www.trolltech.com/products/qt/gplexception/ and
in the file GPL_EXCEPTION.txt in this package.
.
In addition, as a special exception, Trolltech, as the sole copyright
holder for Qt Designer, grants users of the Qt/Eclipse Integration
plug-in the right for the Qt/Eclipse Integration to link to
functionality provided by Qt Designer and its related libraries.
.
Trolltech reserves all rights not expressly granted herein.
.
Trolltech ASA (c) 2007
.
This file is provided AS IS with NO WARRANTY OF ANY KIND, INCLUDING THE
WARRANTY OF DESIGN, MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.
This package is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This package is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this package; if not, write to the Free Software
Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
On Debian systems, the complete text of the GNU General
Public License can be found in `/usr/share/common-licenses/GPL'.
The Debian packaging is (C) 2007, Jean-Francois Dockes <jfd@recoll.org> and
is licensed under the GPL, see above.
Portions of the software are:
Copyright (C) 2001-2002 by Kevin Atkinson under the GNU LGPL:
/* This file is part of The New Aspell
* Copyright (C) 2001-2002 by Kevin Atkinson under the GNU LGPL
* license version 2.0 or 2.1. You should have received a copy of the
* LGPL license along with this library if you did not you can find it
* at http://www.gnu.org/.
On Debian systems, the complete text of the GNU LGPL
can be found in `/usr/share/common-licenses/LGPL-2'.
Copyright 2002-2005 Andreas Aardal Hanssen
Copyright (C) 2000-2004 Mikio Hirabayashi
Copyright 1999,2000,2001 BrightStation PLC
Copyright 2001 Ananova Ltd
Copyright 2002 Olly Betts
Copyright (C) 2000, 2001, 2002 Loic Dachary <loic@senga.org>
- GPL V2 or later, same license text as above
Copyright (c) 1991-2004 Unicode, Inc.
COPYRIGHT AND PERMISSION NOTICE
Copyright © 1991-2006 Unicode, Inc. All rights reserved. Distributed under
the Terms of Use in http://www.unicode.org/copyright.html.
Permission is hereby granted, free of charge, to any person obtaining a
copy of the Unicode data files and any associated documentation (the "Data
Files") or Unicode software and any associated documentation (the
"Software") to deal in the Data Files or Software without restriction,
including without limitation the rights to use, copy, modify, merge,
publish, distribute, and/or sell copies of the Data Files or Software, and
to permit persons to whom the Data Files or Software are furnished to do
so, provided that (a) the above copyright notice(s) and this permission
notice appear with all copies of the Data Files or Software, (b) both the
above copyright notice(s) and this permission notice appear in associated
documentation, and (c) there is clear notice in each modified Data File or
in the Software as well as in the documentation associated with the Data
File(s) or Software that the data or software has been modified.
Files: common/uproplist.h
Copyright: 1991-2006, Unicode, Inc.
License:
All rights reserved. Distributed under the Terms of Use in
http://www.unicode.org/copyright.html
.
Permission is hereby granted, free of charge, to any person obtaining a copy
of the Unicode data files and any associated documentation (the "Data Files")
or Unicode software and any associated documentation (the "Software") to deal
in the Data Files or Software without restriction, including without
limitation the rights to use, copy, modify, merge, publish, distribute, and/or
sell copies of the Data Files or Software, and to permit persons to whom the
Data Files or Software are furnished to do so, provided that (a) the above
copyright notice(s) and this permission notice appear with all copies of the
Data Files or Software, (b) both the above copyright notice(s) and this
permission notice appear in associated documentation, and (c) there is clear
notice in each modified Data File or in the Software as well as in the
documentation associated with the Data File(s) or Software that the data or
software has been modified.
.
THE DATA FILES AND SOFTWARE ARE PROVIDED "AS IS", WITHOUT WARRANTY OF ANY
KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OF
THIRD PARTY RIGHTS. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR HOLDERS
INCLUDED IN THIS NOTICE BE LIABLE FOR ANY CLAIM, OR ANY SPECIAL INDIRECT OR
CONSEQUENTIAL DAMAGES, OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF
USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER
TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
PERFORMANCE OF THE DATA FILES OR SOFTWARE.
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OF THIRD
PARTY RIGHTS. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR HOLDERS INCLUDED IN
THIS NOTICE BE LIABLE FOR ANY CLAIM, OR ANY SPECIAL INDIRECT OR CONSEQUENTIAL
DAMAGES, OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR
PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS
ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THE
DATA FILES OR SOFTWARE.
.
Except as contained in this notice, the name of a copyright holder shall not
be used in advertising or otherwise to promote the sale, use or other dealings
in these Data Files or Software without prior written authorization of the
copyright holder.
Except as contained in this notice, the name of a copyright holder shall
not be used in advertising or otherwise to promote the sale, use or other
dealings in these Data Files or Software without prior written
authorization of the copyright holder.
Files: utils/md5.*
Copyright: 1991-1992, RSA Data Security, Inc. All rights reserved.
License:
MD5C.C - RSA Data Security, Inc., MD5 message-digest algorithm
.
License to copy and use this software is granted provided that it is
identified as the "RSA Data Security, Inc. MD5 Message-Digest Algorithm" in
all material mentioning or referencing this software or this function.
.
License is also granted to make and use derivative works provided that such
works are identified as "derived from the RSA Data Security, Inc. MD5
Message-Digest Algorithm" in all material mentioning or referencing the
derived work.
.
RSA Data Security, Inc. makes no representations concerning either the
merchantability of this software or the suitability of this software for any
particular purpose. It is provided "as is" without express or implied warranty
of any kind.
.
These notices must be retained in any copies of any part of this documentation
and/or software.
Copyright (C) 1991-2, RSA Data Security, Inc. Created 1991. All
/*
* MD5C.C - RSA Data Security, Inc., MD5 message-digest algorithm
*
* Copyright (C) 1991-2, RSA Data Security, Inc. Created 1991. All
* rights reserved.
*
* License to copy and use this software is granted provided that it
* is identified as the "RSA Data Security, Inc. MD5 Message-Digest
* Algorithm" in all material mentioning or referencing this software
* or this function.
*
* License is also granted to make and use derivative works provided
* that such works are identified as "derived from the RSA Data
* Security, Inc. MD5 Message-Digest Algorithm" in all material
* mentioning or referencing the derived work.
*
* RSA Data Security, Inc. makes no representations concerning either
* the merchantability of this software or the suitability of this
* software for any particular purpose. It is provided "as is"
* without express or implied warranty of any kind.
*
* These notices must be retained in any copies of any part of this
* documentation and/or software.
*/
Files: desktop/xdg-utils-1.0.1/*
Copyright: 2006, Kevin Krammer <kevin.krammer@gmx.at>,
Jeremy White <jwhite@codeweavers.com>
License:
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
.
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
Files: index/csguess.cpp
Copyright: 2000-2004, Mikio Hirabayashi
License: LGPL-2.1+
This file is part of QDBM, Quick Database Manager.
.
QDBM is free software; you can redistribute it and/or modify it under
the terms of the GNU Lesser General Public License as published by the
Free Software Foundation; either version 2.1 of the License or any later
version.
.
QDBM is distributed in the hope that it will be useful, but WITHOUT ANY
WARRANTY; without even the implied warranty of MERCHANTABILITY or
FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public
License for more details.
.
You should have received a copy of the GNU Lesser General Public License
along with QDBM; if not, write to the Free Software Foundation, Inc.,
51 Franklin St, Fifth Floor, Boston, MA 02110-1301, USA.
.
On Debian systems, the complete text of the GNU Lesser General Public License
can be found in `/usr/share/common-licenses/LGPL-2.1' and
`/usr/share/common-licenses/LGPL-3'.

View file

@ -1 +0,0 @@
README

View file

@ -1,5 +1,8 @@
setup.py --root and --user options interfer with debian wanting a dist-packages,
not site-packages installation for python modules
Description: Fix python module installation
setup.py --root and --user options interfer with debian wanting a
dist-packages, not site-packages installation for python modules
Author: Jean-Francois Dockes <jf@dockes.org>
Last-Update: 2012-03-26
--- a/recollinstall.in
+++ b/recollinstall.in
@@ -139,5 +139,4 @@

View file

@ -1,7 +1,5 @@
#!/usr/bin/make -f
include /usr/share/quilt/quilt.make
# Uncomment this to turn on verbose mode.
#export DH_VERBOSE=1
@ -28,13 +26,15 @@ config.status: configure
--mandir=\$${prefix}/share/man \
--prefix=/usr
build: build-stamp
build-stamp: $(QUILT_STAMPFN) config.status
build: build-arch build-indep
build-arch: build-stamp
build-indep: build-stamp
build-stamp: config.status
dh_testdir
$(MAKE)
touch $@
clean: unpatch
clean:
dh_testdir
dh_testroot
rm -f build-stamp config.log
@ -46,8 +46,11 @@ install:
dh_testroot
dh_prep
dh_installdirs
$(MAKE) prefix=$(CURDIR)/debian/recoll/usr install
(cd python/recoll;python setup.py install --install-layout=deb --root=$(CURDIR)/debian/recoll/ )
$(MAKE) STRIP=ls prefix=$(CURDIR)/debian/tmp/usr install
(cd python/recoll; python setup.py install \
--install-layout=deb \
--prefix=$(CURDIR)/debian/tmp/usr )
binary-arch: build install
dh_testdir
@ -56,6 +59,7 @@ binary-arch: build install
dh_installdocs README
dh_installmenu
dh_installman
dh_install --sourcedir=debian/tmp
dh_link
dh_strip
dh_compress
@ -69,4 +73,4 @@ binary-arch: build install
binary-indep: build install
binary: binary-indep binary-arch
.PHONY: build clean binary-indep binary-arch binary install
.PHONY: build build-arch build-indep clean binary-indep binary-arch binary install

View file

@ -0,0 +1 @@
3.0 (quilt)

View file

@ -1,12 +1,2 @@
# Example watch control file for uscan
# Rename this file to "watch" and then you can run the "uscan" command
# to check for upstream updates and more.
# See uscan(1) for format
# Compulsory line, this is a version 3 file
version=3
# Uncomment to examine a Webpage
# <Webpage URL> <string match>
http://www.recoll.org/download.html recoll-(.*)\.tar\.gz

View file

@ -235,24 +235,37 @@ More documentation can be found in the doc/ directory or at http://www.recoll.or
files with different character sets, encodings, and languages into the
same index. It has input filters for many document types.
Stemming depends on the document language. Recoll stores the unstemmed
versions of terms and uses auxiliary databases for term expansion. It can
switch stemming languages, or add a language, without re-indexing. Storing
documents in different languages in the same index is possible, and useful
in practice, but does introduce possibilities of confusion. Recoll
currently makes no attempt at automatic language recognition.
Stemming is the process by which Recoll reduces words to their radicals so
that searching does not depend, for example, on a word being singular or
plural (floor, floors), or on a verb tense (flooring, floored). Because
the mechanisms used for stemming depend on the specific grammatical rules
for each language, there is a separate stemmer module for most common
languages where stemming makes sense. Storing documents written in
different languages in the same index is possible, and commonly done. In
this situation, you can specify several stemming languages for the index.
Recoll stores the unstemmed versions of terms in the main index and uses
auxiliary databases for term expansion (one for each stemming language),
which means that you can switch stemming languages between searches, or
add a language without needing a full reindex. Recoll currently makes no
attempt at automatic language recognition, which means that the stemmer
will sometimes be applied to terms from other languages with potentially
strange results. In practise, even if this introduces possibilities of
confusion, this approach has been proven quite useful, and, awaiting the
addition of an automatic language recognition module to Recoll, it is much
less cumbersome than separating your documents according to what language
they are written in.
Recoll has many parameters which define exactly what to index, and how to
classify and decode the source documents. These are kept in configuration
files. A default configuration is copied into a standard location (usually
something like /usr/[local/]share/recoll/examples) during installation.
The default parameters from this file may be overridden by values that you
set inside your personal configuration, found by default in the .recoll
sub-directory of your home directory. The default configuration will index
your home directory with default parameters and should be sufficient for
giving Recoll a try, but you may want to adjust it later, which can be
done either by editing the text files or by using configuration menus in
the recoll GUI
The default values set by the configuration files in this directory may be
overridden by values that you set inside your personal configuration,
found by default in the .recoll sub-directory of your home directory. The
default configuration will index your home directory with default
parameters and should be sufficient for giving Recoll a try, but you may
want to adjust it later, which can be done either by editing the text
files or by using configuration menus in the recoll GUI
Indexing is started automatically the first time you execute the recoll
search graphical user interface, or by executing the recollindex command.

View file

@ -1 +1 @@
1.17.1
1.17.2

View file

@ -109,7 +109,9 @@
#undef _FILE_OFFSET_BITS
#undef _LARGE_FILES
#if _FILE_OFFSET_BITS == 64 || defined(__APPLE__) || defined(__OpenBSD__)
/* The __FreeBSD_kernel__ thing is for debian/kfreebsd. */
#if _FILE_OFFSET_BITS == 64 || defined(__APPLE__) || defined(__OpenBSD__) ||\
defined(__FreeBSD_kernel__)
#define OFFTPC "%lld"
#else
#define OFFTPC "%ld"

View file

@ -107,9 +107,15 @@ RclConfig *recollinit(RclInitFlags flags,
// Make sure the locale charset is initialized (so that multiple
// threads don't try to do it at once).
config->getDefCharset();
// Init unac locking
unac_init_mt();
// Init Unac translation exceptions
string unacex;
if (config->getConfParam("unac_except_trans", unacex) && !unacex.empty())
unac_set_except_translations(unacex.c_str());
int flushmb;
if (config->getConfParam("idxflushmb", &flushmb) && flushmb > 0) {
LOGDEB1(("rclinit: idxflushmb=%d, set XAPIAN_FLUSH_THRESHOLD to 10E6\n",

View file

@ -88,6 +88,7 @@ using namespace std;
#include "unacpp.h"
#include "readfile.h"
#include "rclinit.h"
int main(int argc, char **argv)
{
@ -98,8 +99,13 @@ int main(int argc, char **argv)
}
const char *encoding = argv[1];
string ifn = argv[2];
if (!ifn.compare("stdin"))
ifn.clear();
const char *ofn = argv[3];
string reason;
(void)recollinit(RCLINIT_NONE, 0, 0, reason, 0);
string odata;
if (!file_to_string(ifn, odata)) {
cerr << "file_to_string: " << odata << endl;
@ -111,7 +117,12 @@ int main(int argc, char **argv)
exit(1);
}
int fd = open(ofn, O_CREAT|O_EXCL|O_WRONLY, 0666);
int fd;
if (strcmp(ofn, "stdout")) {
fd = open(ofn, O_CREAT|O_EXCL|O_WRONLY, 0666);
} else {
fd = 1;
}
if (fd < 0) {
cerr << "Open/Create " << ofn << " failed: " << strerror(errno)
<< endl;

28
src/configure vendored
View file

@ -1,6 +1,6 @@
#! /bin/sh
# Guess values for system-dependent variables and create Makefiles.
# Generated by GNU Autoconf 2.68 for Recoll 1.17.0.
# Generated by GNU Autoconf 2.68 for Recoll 1.17.1.
#
#
# Copyright (C) 1992, 1993, 1994, 1995, 1996, 1998, 1999, 2000, 2001,
@ -557,8 +557,8 @@ MAKEFLAGS=
# Identity of this package.
PACKAGE_NAME='Recoll'
PACKAGE_TARNAME='recoll'
PACKAGE_VERSION='1.17.0'
PACKAGE_STRING='Recoll 1.17.0'
PACKAGE_VERSION='1.17.1'
PACKAGE_STRING='Recoll 1.17.1'
PACKAGE_BUGREPORT=''
PACKAGE_URL=''
@ -618,6 +618,7 @@ QTGUI
QMAKE
LIBFAM
LIBXAPIANSTATICEXTRA
LIBXAPIANDIR
LIBXAPIAN
LIBICONV
INCICONV
@ -1257,7 +1258,7 @@ if test "$ac_init_help" = "long"; then
# Omit some internal or obsolete options to make the list less imposing.
# This message is too long to be a string in the A/UX 3.1 sh.
cat <<_ACEOF
\`configure' configures Recoll 1.17.0 to adapt to many kinds of systems.
\`configure' configures Recoll 1.17.1 to adapt to many kinds of systems.
Usage: $0 [OPTION]... [VAR=VALUE]...
@ -1322,7 +1323,7 @@ fi
if test -n "$ac_init_help"; then
case $ac_init_help in
short | recursive ) echo "Configuration of Recoll 1.17.0:";;
short | recursive ) echo "Configuration of Recoll 1.17.1:";;
esac
cat <<\_ACEOF
@ -1449,7 +1450,7 @@ fi
test -n "$ac_init_help" && exit $ac_status
if $ac_init_version; then
cat <<\_ACEOF
Recoll configure 1.17.0
Recoll configure 1.17.1
generated by GNU Autoconf 2.68
Copyright (C) 2010 Free Software Foundation, Inc.
@ -2002,7 +2003,7 @@ cat >config.log <<_ACEOF
This file contains any messages produced by compilers while
running configure, to aid debugging if configure makes a mistake.
It was created by Recoll $as_me 1.17.0, which was
It was created by Recoll $as_me 1.17.1, which was
generated by GNU Autoconf 2.68. Invocation command line was
$ $0 $@
@ -4598,10 +4599,16 @@ done
LIBXAPIAN=$tmpxaplib
# Also recent xapian libs need lz even when they think they don't...
LIBXAPIAN="$LIBXAPIAN -lz"
LIBXAPIANDIR=`$XAPIAN_CONFIG --libs | awk '{print $1}'`
case A"$LIBXAPIANDIR" in
A-L*) LIBXAPIANDIR=`echo $LIBXAPIANDIR | sed -e 's/-L//'`;;
*) LIBXAPIANDIR="";;
esac
XAPIANCXXFLAGS=`$XAPIAN_CONFIG --cxxflags`
#echo XAPIAN_CONFIG: $XAPIAN_CONFIG
#echo LIBXAPIAN: $LIBXAPIAN
#echo LIBXAPIANDIR: $LIBXAPIANDIR
#echo LIBXAPIANSTATICEXTRA: $LIBXAPIANSTATICEXTRA
#echo XAPIANCXXFLAGS: $XAPIANCXXFLAGS
@ -5644,7 +5651,8 @@ test "X$m_prefix" = "XNONE" && m_prefix=/usr/local
m_datadir=${m_prefix}/share
QTRECOLL_DATADIR=${m_datadir}/recoll
RCLVERSION='1.17.0'
RCLVERSION='1.17.1'
@ -6213,7 +6221,7 @@ cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1
# report actual input values of CONFIG_FILES etc. instead of their
# values after options handling.
ac_log="
This file was extended by Recoll $as_me 1.17.0, which was
This file was extended by Recoll $as_me 1.17.1, which was
generated by GNU Autoconf 2.68. Invocation command line was
CONFIG_FILES = $CONFIG_FILES
@ -6275,7 +6283,7 @@ _ACEOF
cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1
ac_cs_config="`$as_echo "$ac_configure_args" | sed 's/^ //; s/[\\""\`\$]/\\\\&/g'`"
ac_cs_version="\\
Recoll config.status 1.17.0
Recoll config.status 1.17.1
configured by $0, generated by GNU Autoconf 2.68,
with options \\"\$ac_cs_config\\"

View file

@ -321,10 +321,16 @@ done
LIBXAPIAN=$tmpxaplib
# Also recent xapian libs need lz even when they think they don't...
LIBXAPIAN="$LIBXAPIAN -lz"
LIBXAPIANDIR=`$XAPIAN_CONFIG --libs | awk '{print $1}'`
case A"$LIBXAPIANDIR" in
A-L*) LIBXAPIANDIR=`echo $LIBXAPIANDIR | sed -e 's/-L//'`;;
*) LIBXAPIANDIR="";;
esac
XAPIANCXXFLAGS=`$XAPIAN_CONFIG --cxxflags`
#echo XAPIAN_CONFIG: $XAPIAN_CONFIG
#echo LIBXAPIAN: $LIBXAPIAN
#echo LIBXAPIANDIR: $LIBXAPIANDIR
#echo LIBXAPIANSTATICEXTRA: $LIBXAPIANSTATICEXTRA
#echo XAPIANCXXFLAGS: $XAPIANCXXFLAGS
@ -504,6 +510,7 @@ AC_SUBST(X_EXTRA_LIBS)
AC_SUBST(INCICONV)
AC_SUBST(LIBICONV)
AC_SUBST(LIBXAPIAN)
AC_SUBST(LIBXAPIANDIR)
AC_SUBST(LIBXAPIANSTATICEXTRA)
AC_SUBST(LIBFAM)
AC_SUBST(QMAKE)

View file

@ -0,0 +1,212 @@
/*
* Copyright (c) 2001, 2003, 2010 The FreeBSD Documentation Project
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
* SUCH DAMAGE.
*
* $FreeBSD: doc/share/misc/docbook.css,v 1.15 2010/03/20 04:15:01 hrs Exp $
*/
body address {
line-height: 1.3;
margin: .6em 0;
}
body blockquote {
margin-top: .75em;
line-height: 1.5;
margin-bottom: .75em;
}
html body {
margin: 1em 8% 1em 10%;
line-height: 1.2;
}
.legalnotice {
font-size: small;
font-variant: small-caps;
}
body div {
margin: 0;
}
dl {
margin: .8em 0;
line-height: 1.2;
}
body form {
margin: .6em 0;
}
h1, h2, h3, h4, h5, h6,
div.example p b,
.question,
div.table p b,
div.procedure p b {
color: #990000;
}
body h1, body h2, body h3, body h4, body h5, body h6 {
line-height: 1.3;
margin-left: 0;
}
body h1, body h2 {
margin: .8em 0 0 -4%;
}
body h3, body h4 {
margin: .8em 0 0 -3%;
}
body h5 {
margin: .8em 0 0 -2%;
}
body h6 {
margin: .8em 0 0 -1%;
}
body hr {
margin: .6em;
border-width: 0 0 1px 0;
border-style: solid;
border-color: #cecece;
}
body img.navheader {
margin: 0 0 0 -4%;
}
ol {
margin: 0 0 0 5%;
line-height: 1.2;
}
body pre {
margin: .75em 0;
line-height: 1.0;
font-family: monospace;
}
body td, body th {
line-height: 1.2;
}
ul, body dir, body menu {
margin: 0 0 0 5%;
line-height: 1.2;
}
html {
margin: 0;
padding: 0;
}
body p b.application {
color: #000000;
}
body p span.application {
font-weight: bold;
color: #000000;
}
.filename {
color: #007a00;
}
.guimenu, .guimenuitem, .guisubmenu,
.guilabel, .interface,
.shortcut, .shortcut .keycap {
font-weight: bold;
}
.guibutton {
background-color: #cfcfcf;
padding: 2px;
}
.accel {
background-color: #f0f0f0;
text-decoration: underline;
}
.screen {
padding: 1ex;
}
.programlisting {
padding: 1ex;
background-color: #eee;
border: 1px solid #ccc;
}
@media screen { /* hide from ie3 */
a[href]:hover { background: #ffa }
}
blockquote.note {
color: #222;
background: #eee;
border: 1px solid #ccc;
padding: 0.4em 0.4em;
width: 85%;
}
blockquote.tip {
color: #004f00;
background: #d8ecd6;
border: 1px solid green;
padding: 0.2em 2em;
width: 85%;
}
blockquote.important {
font-style:italic;
border: 1px solid #a00;
border-left: 12px solid #c00;
padding: 0.1em 1em;
}
blockquote.warning {
color: #9f1313;
background: #f8e8e8;
border: 1px solid #e59595;
padding: 0.2em 2em;
width: 85%;
}
.example {
background: #fefde6;
border: 1px solid #f1bb16;
margin: 1em 0;
padding: 0.2em 2em;
width: 90%;
}
.informaltable table.calstable tr td {
padding-left: 1em;
padding-right: 1em;
}

View file

@ -34,6 +34,9 @@
<para>This document introduces full text search notions
and describes the installation and use of the &RCL;
application. It currently describes &RCL; &RCLVERSION;.</para>
<!-- <para>[ <ulink url="index.html">Split HTML</ulink> /
<ulink url="usermanual-xml.html">Single HTML</ulink> ]</para>
-->
</abstract>
@ -45,12 +48,12 @@
<sect1 id="rcl.introduction.tryit">
<title>Giving it a try</title>
<para>If you do not like reading manuals (who does?) and would
like to give &RCL; a try, just perform <link
linkend="rcl.install.binary">installation</link> and start the
<command>recoll</command> user interface, which will index your
home directory by default, allowing you to search immediately after
indexing completes.</para>
<para>If you do not like reading manuals (who does?) and would like
to give &RCL; a try, just <link
linkend="rcl.install.binary">install</link> the application and
start the <command>recoll</command> graphical user interface (GUI),
which will ask to index your home directory by default, allowing
you to search immediately after indexing completes.</para>
<para>Do not do this if your home directory contains a huge
number of documents and you do not want to wait or are very
@ -135,45 +138,65 @@
different character sets, encodings, and languages into the same
index. It has input filters for many document types.</para>
<para>Stemming depends on the document language. &RCL; stores
the unstemmed versions of terms and uses auxiliary databases for
term expansion. It can switch stemming languages, or add a
language, without re-indexing. Storing documents in different
languages in the same index is possible, and useful in
practice, but does introduce possibilities of confusion. &RCL;
currently makes no attempt at automatic language recognition.</para>
<para>Stemming is the process by which &RCL; reduces words to
their radicals so that searching does not depend, for example,
on a word being singular or plural (floor, floors), or on a verb
tense (flooring, floored). Because the mechanisms used for
stemming depend on the specific grammatical rules for each
language, there is a separate stemmer module for most common
languages where stemming makes sense. Storing documents written
in different languages in the same index is possible, and
commonly done. In this situation, you can specify several
stemming languages for the index. &RCL; stores the unstemmed
versions of terms in the main index and uses auxiliary databases
for term expansion (one for each stemming language), which means
that you can switch stemming languages between searches, or add
a language without needing a full reindex. &RCL; currently
makes no attempt at automatic language recognition, which means
that the stemmer will sometimes be applied to terms from other
languages with potentially strange results. In practise, even if
this introduces possibilities of confusion, this approach has
been proven quite useful, and, awaiting the addition of an
automatic language recognition module to &RCL;, it is much less
cumbersome than separating your documents according to what
language they are written in.</para>
<para>&RCL; has many parameters which define exactly what to
index, and how to classify and decode the source documents. These
are kept in <link linkend="rcl.indexing.config">configuration
files</link>. A default configuration is copied into a standard
location (usually something like
<filename>/usr/[local/]share/recoll/examples</filename>) during
installation. The default parameters from this file may be
overridden by values that you set inside your personal
configuration, found by default in the <filename>.recoll</filename>
sub-directory of your home directory. The default configuration
will index your home directory with default parameters and should
be sufficient for giving &RCL; a try, but you may want to adjust it
later, which can be done either by editing the text files or by
using configuration menus in the <command>recoll</command>
GUI</para>
index, and how to classify and decode the source
documents. These are kept in <link
linkend="rcl.indexing.config">configuration files</link>. A
default configuration is copied into a standard location
(usually something like
<filename>/usr/[local/]share/recoll/examples</filename>)
during installation. The default values set by the
configuration files in this directory may be overridden by
values that you set inside your personal configuration, found
by default in the <filename>.recoll</filename> sub-directory
of your home directory. The default configuration will index
your home directory with default parameters and should be
sufficient for giving &RCL; a try, but you may want to adjust
it later, which can be done either by editing the text files
or by using configuration menus in the
<command>recoll</command> GUI</para>
<para><link linkend="rcl.indexing.periodic.exec">Indexing</link>
is started automatically the first time you execute the
<command>recoll</command> search graphical user interface, or by
executing the <command>recollindex</command> command.</para>
<para>The <link linkend="rcl.indexing.periodic.exec">indexing
process</link> is started automatically the first time you
execute the <command>recoll</command> GUI. Indexing can also be
performed by executing the <command>recollindex</command>
command.</para>
<para><link linkend="rcl.search">Searches</link> are usually
performed inside the <command>recoll</command> graphical user
interface (GUI) program, which has many options to help you find
what you are looking for. However, there are other ways to perform
&RCL; searches: mostly a <link linkend="rcl.search.commandline">
command line tool</link>, a
performed inside the <command>recoll</command> GUI, which has many
options to help you find what you are looking for. However, there
are other ways to perform &RCL; searches: mostly a <link
linkend="rcl.search.commandline">
command line interface</link>, a
<link linkend="rcl.program.api.python">
<application>Python</application>
programming interface</link>, and a <link linkend="rcl.searchkio">
<application>KDE</application> KIO slave module</link>.</para>
programming interface</link>, a <link linkend="rcl.searchkio">
<application>KDE</application> KIO slave module</link>, and
a <application>Ubuntu Unity Lens</application> module.
</para>
</sect1>
</chapter>
@ -234,25 +257,27 @@
<link linkend="rcl.indexing.config">configuration files</link>.</para>
<para>Most file types, like HTML or word processing files, only hold
one document. Some file types, like mail folder files or zip
one document. Some file types, like email folders or zip
archives, can hold many individually indexed documents, which may
in turn be themselves compound ones. Such hierarchies can go quite
deep, and &RCL; has no problem processing, for example, an ms-word
document which would be an attachment to an email message part of
a folder file archived inside a zip file...</para>
deep, and &RCL; can process, for example, an
<application>ms-word</application>
document stored as an attachment to an email message inside an
email folder archived in a zip file...</para>
<para>&RCL; indexing processes plain text, HTML, openoffice
and e-mail files, and a few others internally.</para>
<para>&RCL; indexing processes plain text, HTML, OpenDocument
(Open/LibreOffice), email formats, and a few others internally.</para>
<para>Other file types (ie: postscript, pdf, ms-word, rtf ...)
need external applications for preprocessing. The list is in the
<link linkend="rcl.install.external"> installation</link>
section. After every indexing operation, &RCL; updates a list of
commands that would be needed for indexing existing files
types. This list can be displayed from the
<command>recoll</command> <guilabel>File</guilabel> menu. It is
stored in the <filename>missing</filename> text file
inside the configuration directory.</para>
types. This list can be displayed by selecting the menu option
<guilabel>File</guilabel>-><guilabel>Show Missing Helpers</guilabel>
in the <command>recoll</command> GUI. It is stored in the
<filename>missing</filename> text file inside the configuration
directory.</para>
<para>Without further configuration, &RCL; will index all
appropriate files from your home directory, with a reasonable
@ -336,9 +361,9 @@ recoll
indexed).</para>
<para>Of course, images, sound and video do not increase the
index size, which means that it will be quite typical nowadays
(2006), that even a big index will be negligible against the
total amount of data on the computer.</para>
index size, which means that nowadays (2012), typically, even a big
index will be negligible against the total amount of data on the
computer.</para>
<para>The index data directory (<filename>xapiandb</filename>)
only contains data that can be completely rebuilt by an index run
@ -439,14 +464,20 @@ recoll
option.)</para>
<para>The interface is started from the
<guilabel>Preferences</guilabel> menu. It has two main
panels. The first panel allows setting global variables, like
the list of top directories or the list of skipped paths. The
second panel allows setting variables that can be redefined
for subdirectories. This second panel has an initially empty list of
customisation directories, to which you can add. The variables
are then set for the currently selected directory (or at the top
level if the empty line is selected).</para>
<guilabel>Preferences</guilabel>-><guilabel>Indexing
Configuration</guilabel> menu entry. It is divided in three tabs,
<guilabel>Global parameters</guilabel>, <guilabel>Local
parameters</guilabel>, and <guilabel>Beagle web history</guilabel>,
which is explained in the next section.</para>
<para>The first tab allows setting global variables, like the lists
of top directories, skipped paths, or stemming languages.</para>
<para>The second tab allows setting variables that can be redefined
for subdirectories. This second tab has an initially empty list of
customisation directories, to which you can add. The variables are
then set for the currently selected directory (or at the top level
if the empty line is selected).</para>
<para>The meaning for most entries in the interface is
self-evident and documented by a <literal>ToolTip</literal>
@ -521,15 +552,17 @@ recoll
if canceled).</para>
<para>The <command>recollindex</command> indexing process can be
interrupted by sending an interrupt (^C, SIGINT) or terminate
interrupted by sending an interrupt (Ctrl-C, SIGINT) or terminate
(SIGTERM) signal. Some time may elapse before the process exits,
because it needs to properly flush and close the index. The
indexing thread can be equivalently stopped from the menu.</para>
because it needs to properly flush and close the index. This can
also be done from the <command>recoll</command> GUI
<guilabel>File</guilabel>-><guilabel>Stop Indexing</guilabel>
menu entry.</para>
<para>After such an interruption, the index will be somewhat
inconsistent because some operations which are normally performed
at the end of the indexing pass will have been skipped (for
exemple, the stemming and spelling databases will be inexistant
example, the stemming and spelling databases will be inexistant
or out of date). You just need to restart indexing at a later
time to restore consistency. The indexing will restart at the
interruption point (the full file tree will be traversed,
@ -576,7 +609,8 @@ recoll
<para>As of version 1.17 the &RCL; GUI has dialogs to manage
<filename>crontab</filename> entries for
<command>recollindex</command>. You can reach them from the
<guimenu>Preferences->Indexing Schedule</guimenu> menu. They only
<guilabel>Preferences</guilabel>-><guilabel>Indexing
Schedule</guilabel> menu. They only
work with the good old <command>cron</command>, and do not give
access to all features of <command>cron</command> scheduling.</para>
@ -652,12 +686,13 @@ fvwm
on the log level.</para>
<para>When building &RCL;, the real time indexing support can be
customised during package
<link linkend="rcl.install.building.build">configuration</link>
with the <literal>--with[out]-fam</literal> or
customised during package <link
linkend="rcl.install.building.build">configuration</link> with the
<literal>--with[out]-fam</literal> or
<literal>--with[out]-inotify</literal> options. The default is
currently to include inotify monitoring on systems that support
it, and, as of recoll 1.17, gamin support on FreeBSD.</para>
currently to include <application>inotify</application> monitoring
on systems that support it, and, as of recoll 1.17,
<application>gamin</application> support on FreeBSD.</para>
<para>While it is convenient that data is indexed in real time,
repeated indexing can generate a significant load on the
@ -712,7 +747,7 @@ fvwm
<para>In most cases, you can enter the terms as you
think them, even if they contain embedded punctuation or other
non-textual characters. For
exemple, &RCL; can handle things like e-mail addresses, or
example, &RCL; can handle things like email addresses, or
arbitrary cut and paste from another text window, punctation
and all.</para>
@ -950,7 +985,7 @@ fvwm
that you can't actually visualize the folder (there will be an
error dialog if you try). &RCL; is unfortunately not yet smart
enough to disable the entry in this case. In other cases, the
<guilabel>Open</guilabel> option makes sense, for exemple to
<guilabel>Open</guilabel> option makes sense, for example to
start a <application>chm</application> viewer on the parent
document for a help page.</para>
@ -1006,7 +1041,7 @@ fvwm
create a new preview window. The old one stays open until you
close it.</para>
<para>You can close a preview tab by typing <keycap>^W</keycap>
<para>You can close a preview tab by typing <keycap>Ctrl-W</keycap>
(<keycap>Ctrl</keycap> + <keycap>W</keycap>) in the
window. Closing the last tab for a window will also close the
window.</para>
@ -1030,7 +1065,7 @@ fvwm
<keycap>F3</keycap> inside the text area to get to the next
occurrence.</para>
<para>If you have a search string entered and you use ^Up/^Down
<para>If you have a search string entered and you use Ctrl-Up/Ctrl-Down
to browse the results, the search is initiated for each successive
document. If the string is found, the cursor will be positioned
at the first occurrence of the search string.</para>
@ -1042,8 +1077,8 @@ fvwm
the main text but in one of the fields.</para>
<para>You can print the current preview window contents by typing
<keycap>^P</keycap> (<keycap>Ctrl</keycap> + <keycap>P</keycap>) in
the window text.</para>
<keycap>Ctrl-P</keycap> (<keycap>Ctrl</keycap> +
<keycap>P</keycap>) in the window text.</para>
</sect2>
@ -1539,19 +1574,19 @@ fvwm
</formalpara>
<formalpara><title>Closing previews</title>
<para>Entering <keycap>^W</keycap> in a tab will
<para>Entering <keycap>Ctrl-W</keycap> in a tab will
close it (and, for the last tab, close the preview
window). Entering <keycap>Esc</keycap> will close the preview
window and all its tabs.</para>
</formalpara>
<formalpara><title>Printing previews</title>
<para>Entering <keycap>^P</keycap> in a preview window will print
<para>Entering <keycap>Ctrl-P</keycap> in a preview window will print
the currently displayed text.</para>
</formalpara>
<formalpara><title>Quitting</title>
<para>Entering <keycap>^Q</keycap> almost anywhere will
<para>Entering <keycap>Ctrl-Q</keycap> almost anywhere will
close the application.</para>
</formalpara>
</sect3>
@ -1588,9 +1623,10 @@ fvwm
on startup. The default value is empty, but there is a
skeleton style sheet (<filename>recoll.qss</filename>)
inside the <filename>/usr/share/recoll/examples</filename>
directory. Using a style sheet, you can change most Recoll
graphical parameters: colors, fonts, etc. See the sample
file for a few simple examples.</para>
directory. Using a style sheet, you can change most
<command>recoll</command> graphical parameters: colors,
fonts, etc. See the sample file for a few simple
examples.</para>
</listitem>
<listitem><para><guilabel>Maximum text size highlighted for
@ -1830,7 +1866,7 @@ fvwm
<para>No more detail will be given about the header part (only
useful with the WebKit build), if there are restrictions to
what you can do, they are beyond this author's HTML/CSS/Javascript
abilities... There are a few exemples on the
abilities... There are a few examples on the
<ulink url="http://www.recoll.org/custom.html">page about
customising the result list</ulink> on the &RCL; web site.</para>
@ -2126,7 +2162,7 @@ text/html [file:///Users/uncrypted-dockes/projets/bateaux/ilur/factEtCie/r
<replaceable>potatoes</replaceable> (in any part of the document).</para>
<para>An element is composed of an optional field specification,
and a value, separated by a colon. Exemple:
and a value, separated by a colon. Example:
<replaceable>Beatles</replaceable>,
<replaceable>author:balzac</replaceable>,
<replaceable>dc:title:grandet</replaceable> </para>
@ -2163,7 +2199,7 @@ text/html [file:///Users/uncrypted-dockes/projets/bateaux/ilur/factEtCie/r
<replaceable>title:prejudice title:pride</replaceable>, and is
unlikely to find a result.</para>
<para>Modifiers can be set on a phrase clause, for exemple to specify
<para>Modifiers can be set on a phrase clause, for example to specify
a proximity search (unordered). See
<link linkend="rcl.search.lang.modifiers">the modifier
section</link>.</para>
@ -2209,7 +2245,7 @@ text/html [file:///Users/uncrypted-dockes/projets/bateaux/ilur/factEtCie/r
</listitem>
<listitem><para><literal>size</literal> for filtering the
results on file size. Exemple:
results on file size. Example:
<literal>size&lt;10000</literal>. You can use
<literal>&lt;</literal>, <literal>&gt;</literal> or
<literal>=</literal> as operators. You can specify a range like the
@ -2233,7 +2269,7 @@ text/html [file:///Users/uncrypted-dockes/projets/bateaux/ilur/factEtCie/r
The days and months parts may be missing. If the
<literal>/</literal> is present but an element is missing, the
missing element is interpreted as the lowest or highest date in the
index. Exemples:</para>
index. Examples:</para>
<itemizedlist>
<listitem><para><literal>2001-03-01/2002-05-01</literal> the
basic syntax for an interval of dates.</para>
@ -2555,7 +2591,7 @@ text/html [file:///Users/uncrypted-dockes/projets/bateaux/ilur/factEtCie/r
<literal>Subject:</literal> for email) when indexing. This is not
essential.</para>
<para>You should look to one of the simple filters, for exemple
<para>You should look to one of the simple filters, for example
<literal>rclps</literal> for a starting point.</para>
<para>Don't forget to make your filter executable before
@ -3087,7 +3123,7 @@ while query.next >= 0 and query.next < nres:
<para>You will only have to check or install <link
linkend="rcl.install.external">supporting applications</link>
for the file types that you want to index beyond those that are
natively processed by &RCL; (text, HTML, mail files, and a few
natively processed by &RCL; (text, HTML, email files, and a few
others).</para>
<para>You should also maybe have a look at the
@ -3259,13 +3295,13 @@ while query.next >= 0 and query.next < nres:
<listitem><para>Konqueror webarchive format with Python (uses the
Tarfile module).</para></listitem>
<listitem><para>mimehtml web archive format (support based on the mail
<listitem><para>mimehtml web archive format (support based on the email
filter, which introduces some mild weirdness, but still
usable).</para></listitem>
</itemizedlist>
<para>Text, HTML, mail folders, and Scribus files are
<para>Text, HTML, email folders, and Scribus files are
processed internally. <application>Lyx</application> is used to
index Lyx files. Many filters need <command>iconv</command> and the
standard <command>sed</command> and <command>awk</command>.
@ -3557,6 +3593,28 @@ while query.next >= 0 and query.next < nres:
List elements with embedded spaces can be quoted using
double-quotes.</para>
<formalpara><title>Encoding issues</title>
<para>Most of the configuration parameters are plain ASCII. Two
particular sets of values may cause encoding issues:</para>
<itemizedlist>
<listitem><para>File path parameters may contain non-ascii
characters and should use the exact same byte values as found in
the file system directory. Usually, this means that the
configuration file should use the system default locale
encoding.</para>
</listitem>
<listitem><para>The <literal>unac_except_trans</literal> parameter
should be encoded in UTF-8. If your system locale is not UTF-8, and
you need to also specify non-ascii file paths, this poses a
difficulty because common text editors cannot handle multiple
encodings in a single file. In this relatively unlikely case, you
can edit the configuration file as two separate text files with
appropriate encodings, and concatenate them to create the complete
configuration.</para>
</listitem>
</itemizedlist>
<sect2 id="rcl.install.config.recollconf">
<title>Main configuration file</title>
@ -3611,7 +3669,7 @@ skippedNames = #* bin CVS Cache cache* caughtspam tmp .thumbnails .svn \
<para>The list in the default configuration does not
exclude hidden directories (names beginning with a
dot), which means that it may index quite a few things
that you do not want. On the other hand, mail user
that you do not want. On the other hand, email user
agents like <application>thunderbird</application>
usually store messages in hidden directories, and you
probably want this indexed. One possible solution is to
@ -3816,9 +3874,36 @@ skippedPaths = ~/somedir/&lowast;.txt
</listitem>
</varlistentry>
<varlistentry><term><literal>unac_except_trans</literal></term>
<listitem><para>This is a list of characters, encoded in UTF-8,
which should be handled specially when converting text to
unaccented lowercase. For example, in Swedish, the letter
<literal>a with diaeresis </literal> has full alphabet
citizenship and should not be turned into an
<literal>a</literal>. Each element in the space-separated list
has the special character as first element and the translation
following. The handling of both the lowercase and upper-case
versions of a character should be specified, as appartenance to
the list will turn-off both standard accent and case
processing. Example for Swedish:</para>
<programlisting>
unac_except_trans = åå Åå ää Ää öö Öö
</programlisting>
<para>Note that the translation is not limited to a single
character, you could very well have something like
<literal>üue</literal> in the list.</para>
<para>This parameter can't be defined for subdirectories, it
is global, because there is no way to do otherwise when
querying. If you have document sets which would need different
values, you will have to index and query them separately.</para>
</listitem>
</varlistentry>
<varlistentry><term><literal>maildefcharset</literal></term>
<listitem><para>This can be used to define the default
character set specifically for mail messages which don't
character set specifically for email messages which don't
specify it. This is mainly useful for readpst (libpst) dumps,
which are utf-8 but do not say so.</para>
</listitem>
@ -4081,9 +4166,9 @@ mondelaypatterns = *.log:20 "this one has spaces*:10"
<varlistentry>
<term>filter-specific sections</term>
<listitem><para>Some filters may need specific
configuration for handling fields. Only the mail message filter
configuration for handling fields. Only the email message filter
currently has such a section (named
<literal>[mail]</literal>). It allows indexing arbitrary mail
<literal>[mail]</literal>). It allows indexing arbitrary email
headers in addition to the ones indexed by default. Other such
sections may appear in the future.</para>
</listitem>
@ -4093,9 +4178,9 @@ mondelaypatterns = *.log:20 "this one has spaces*:10"
<para>Here follows a small example of a personal
<filename>fields</filename>
file. This would extract a specific mail header and
file. This would extract a specific email header and
use it as a searchable field, with data displayable inside result
lists. (Side note: as the mail filter does no decoding on the values,
lists. (Side note: as the email filter does no decoding on the values,
only plain ascii headers can be indexed, and only the
first occurrence will be used for headers that occur several times).

49
src/doc/user/xmlmake.sh Normal file
View file

@ -0,0 +1,49 @@
#!/bin/sh
# A script to produce the Recoll manual with an xml toolchain.
# Limitations:
# - Does not produce the links to the whole/chunked versions at the top
# of the document
# - The anchor names from the source text are converted to uppercase by
# the sgml toolchain. This does not happen with the xml toolchain,
# which means that external links like
# usermanual.html#RCL.CONFIG.INDEXING won't work because fragments are
# case-sensitive. This could be solved by converting all ids inside the
# source file to upper-case.
# Wherever docbook.xsl and chunk.xsl live
XSLDIR="/usr/local/share/xsl/docbook/"
dochunky=1
test $# -eq 1 && dochunky=0
# Remove the SGML header and uncomment the XML one
sed -e '\!//FreeBSD//DTD!d' \
-e '\!DTD DocBook XML!s/<!--//' \
-e '\!/docbookx.dtd!s/-->//' \
< usermanual.sgml > usermanual.xml
# Options common to the single-file and chunked versions
commonoptions="--stringparam section.autolabel 1 \
--stringparam section.autolabel.max.depth 3 \
--stringparam section.label.includes.component.label 1 \
--stringparam autotoc.label.in.hyperlink 0 \
--stringparam abstract.notitle.enabled 1 \
--stringparam html.stylesheet docbook-xsl.css \
--stringparam generate.toc \"book toc,title,figure,table,example,equation\" \
"
# Do the chunky thing
test $dochunky -ne 0 && eval xsltproc $commonoptions \
--stringparam use.id.as.filename 1 \
"$XSLDIR/html/chunk.xsl" \
usermanual.xml
# Produce the single file version
eval xsltproc $commonoptions \
-o usermanual-xml.html \
"$XSLDIR/html/docbook.xsl" \
usermanual.xml
tidy -indent usermanual-xml.html > tmpfile
mv -f tmpfile usermanual-xml.html

View file

@ -2,11 +2,20 @@
"""Extract Html files from a Microsoft Compiled Html Help file (.chm)
Needs at least python 2.2 for HTMLParser (chmlib needs 2.2 too)"""
# Do we return individual chapters as html pages or concatenate everything?
rclchm_catenate = 0
# Use special html type to allow for mimeconf/mimeview Open magic,
# Or go the regular html way with text/html
#rclchm_html_mtype = "text/x-chm-html"
rclchm_html_mtype = "text/html"
import sys
import os
import posixpath
import urlparse
import urllib
if rclchm_catenate:
import subprocess
import rclexecm
@ -171,6 +180,10 @@ class rclCHM:
self.tp = ChmTopicsParser(em)
self.currentindex = 0
self.em = em
if rclchm_catenate:
self.em.setmimetype("text/plain")
else:
self.em.setmimetype(rclchm_html_mtype)
def extractone(self, path):
"""Extract one path-named internal file from the chm file"""
@ -188,10 +201,27 @@ class rclCHM:
res, doc = self.chm.RetrieveObject(ui)
#self.em.rclog("extract: RetrieveObject: %d [%s]" % (res, doc))
if res > 0:
self.em.setmimetype("text/html")
self.em.setmimetype(rclchm_html_mtype)
return (True, doc, path, iseof)
return (False, "", path, iseof)
def dumpall(self):
alltxt=""
for pth in self.tp.contents:
ret,doc,path,iseof = self.extractone(pth)
if not ret:
continue
# Feed doc to lynx
process = subprocess.Popen(["lynx", "-stdin", "-dump", "-nolist",
"-display_charset=utf8",
"-force_html"],
stdin=subprocess.PIPE,
stdout=subprocess.PIPE
)
txt,err = process.communicate(doc)
alltxt += txt
return alltxt
def openfile(self, params):
"""Open the chm file and build the contents list by extracting and
parsing the Topics object"""
@ -239,6 +269,13 @@ class rclCHM:
return self.extractone(params["ipath:"])
def getnext(self, params):
if rclchm_catenate:
alltxt = self.dumpall()
if alltxt:
return (True, alltxt, "", rclexecm.RclExecM.eofnext)
else:
return (False, "", "", rclexecm.RclExecM.eofnow)
if self.currentindex >= len(self.tp.contents):
return (False, "", "", rclexecm.RclExecM.eofnow)
else:

124
src/filters/rcldia Executable file
View file

@ -0,0 +1,124 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# dia (http://live.gnome.org/Dia) file filter for recoll
# stefan.friedel@iwr.uni-heidelberg.de 2012
#
# add the following to ~/.recoll/mimeconf into the [index] section:
# application/x-dia-diagram = execm rcldia;mimetype=text/html;charset=utf-8
# and into the [icons] section:
# application/x-dia-diagram = drawing
# and finally under [categories]:
# other = ...\
# application/x-dia-diagram
#
# in ~/.recoll/mimemap:
# .dia = application/x-dia-diagram
# Small fixes from jfd: dia files are sometimes not compressed.
# And a note: this file actually has no reason to return HTML as there is
# no metadata. We could just as well and more simply return text/plain
import rclexecm
import re
from gzip import GzipFile
import xml.parsers.expat
# some regexps to parse/format the xml data: delete #/spaces at the b/eol and
# ignore empty lines
rhs = re.compile(r'^[#|\s+](.*)')
rhe = re.compile(r'(.*)[#|\s+]$')
rempty = re.compile(r'^#?\s*#?$')
htmltemplate = '''
<html><head>
<meta http-equiv="Content-Type" content="text/html;charset=UTF-8">
</head>
<body>
{0}
</body>
</html>
'''
# xml parser for dia xml file
class Parser:
def __init__(self,rclem):
self._parser = xml.parsers.expat.ParserCreate(encoding='UTF-8')
self._parser.StartElementHandler = self.startelement
self._parser.EndElementHandler = self.endelement
self._parser.CharacterDataHandler = self.chardata
self.string = []
self.handlethis = False
self.rclem = rclem
def startelement(self, name, attrs):
if name == 'dia:string':
self.handlethis = True
else:
self.handlethis = False
def chardata(self,data):
if self.handlethis:
# check if line is not empty and replace hashes/spaces
# tricky: after htmlescape check also for umlauts
if not rempty.search(data):
self.string.append(self.rclem.htmlescape(
rhe.sub(r'\1',rhs.sub(r'\1',data))).encode('ascii', 'xmlcharrefreplace'))
def endelement(self,name):
self.handlethis = False
def feed(self, fh):
self._parser.ParseFile(fh)
del self._parser
class DiaExtractor:
def __init__(self, em):
self.em = em
def extractdia(self):
docdata = ""
ipath = ""
try:
docdata = self.ExtractDiaText()
ok = True
except Exception, err:
ok = False
iseof = rclexecm.RclExecM.eofnext
self.em.setmimetype("text/html")
return (ok, docdata, ipath, iseof)
###### File type handler api, used by rclexecm ---------->
def openfile(self, params):
try:
self.dia = GzipFile(params["filename:"], 'r')
# Dial files are sometimes not compressed. Quite weirdly,
# GzipFile does not complain until we try to read. Have to do it
# here to be able to retry an uncompressed open.
data = self.dia.readline()
self.dia.seek(0)
return True
except:
# File not compressed ?
try:
self.dia = open(params["filename:"], 'r')
except:
return False
return True
def getipath(self, params):
ok, data, ipath, eof = self.extractdia()
return (ok, data, ipath, eof)
def getnext(self, params):
ok, data, ipath, eof = self.extractdia()
return (ok, data, ipath, eof)
###### read data
def ExtractDiaText(self):
diap = Parser(self.em)
diap.feed(self.dia)
return htmltemplate.format('\n'.join(diap.string))
# Main program: create protocol handler and extractor and run them
proto = rclexecm.RclExecM()
extract = DiaExtractor(proto)
rclexecm.main(proto, extract)

View file

@ -24,8 +24,11 @@ class RclExecM:
sys.exit(exitvalue)
def htmlescape(self, txt):
txt = txt.replace("<", "&lt;")
# This must stay first (it somehow had managed to skip after
# the next line, with rather interesting results)
txt = txt.replace("&", "&amp;")
txt = txt.replace("<", "&lt;")
txt = txt.replace('"', "&dquot;")
return txt

View file

@ -28,14 +28,6 @@ subtreelist : $(SUBTREELIST_OBJS)
subtreelist.o : subtreelist.cpp
$(CXX) $(ALL_CXXFLAGS) -DTEST_SUBTREELIST -c subtreelist.cpp
CSGUESS_OBJS= trcsguess.o $(BIGLIB)
csguess : $(CSGUESS_OBJS)
$(CXX) $(ALL_CXXFLAGS) -o csguess $(CSGUESS_OBJS) \
$(LIBICONV)
trcsguess.o : csguess.cpp
$(CXX) $(ALL_CXXFLAGS) -DTEST_CSGUESS -c -o trcsguess.o \
csguess.cpp
MIMETYPE_OBJS= trmimetype.o $(BIGLIB)
mimetype : $(MIMETYPE_OBJS)
$(CXX) $(ALL_CXXFLAGS) -o mimetype $(MIMETYPE_OBJS) \

View file

@ -1,215 +0,0 @@
/* Copyright (C) 2004 J.F.Dockes
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the
* Free Software Foundation, Inc.,
* 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
*/
#ifndef TEST_CSGUESS
// This code was converted from estraier / qdbm / myconf.c:
/**************************************************************************
* Copyright (C) 2000-2004 Mikio Hirabayashi
*
* This file is part of QDBM, Quick Database Manager.
*
* QDBM is free software; you can redistribute it and/or modify it under the
* terms of the GNU Lesser General Public License as published by the Free
* Software Foundation; either version 2.1 of the License or any later
* version. QDBM is distributed in the hope that it will be useful, but
* WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
* or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public
* License for more details. You should have received a copy of the GNU
* Lesser General Public License along with QDBM; if not, write to the Free
* Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA
* 02111-1307 USA.
* *********************************************************/
#include <errno.h>
#include <cstring>
#include <iostream>
#ifndef NO_NAMESPACES
using std::string;
#endif /* NO_NAMESPACES */
#include <iconv.h>
#include "csguess.h"
#include "autoconfig.h"
#ifdef RCL_ICONV_INBUF_CONST
#define ICV_P2_TYPE const char**
#else
#define ICV_P2_TYPE char**
#endif
// The values from estraier were 32768, 256, 0.001
const int ICONVCHECKSIZ = 32768;
const int ICONVMISSMAX = 256;
const double ICONVALLWRAT = 0.001;
// Try to transcode and count errors (for charset guessing)
static int transcodeErrCnt(const char *ptr, int size,
const char *icode, const char *ocode)
{
iconv_t ic;
char obuf[2*ICONVCHECKSIZ], *wp, *rp;
size_t isiz, osiz;
int miss;
isiz = size;
if((ic = iconv_open(ocode, icode)) == (iconv_t)-1)
return size;
miss = 0;
rp = (char *)ptr;
while(isiz > 0){
osiz = 2*ICONVCHECKSIZ;
wp = obuf;
if(iconv(ic, (ICV_P2_TYPE)&rp, &isiz, &wp, &osiz) == (size_t)-1){
if(errno == EILSEQ || errno == EINVAL){
rp++;
isiz--;
miss++;
if(miss >= ICONVMISSMAX)
break;
} else {
miss = size;
break;
}
}
}
if(iconv_close(ic) == -1)
return size;
return miss;
}
// Try to guess character encoding. This could be optimized quite a
// lot by avoiding the multiple passes on the document, to be done
// after usefulness is demonstrated...
string csguess(const string &in, const string &dflt)
{
const char *hypo;
int i, miss;
const char *text = in.c_str();
bool cr = false;
int size = in.length();
if (size > ICONVCHECKSIZ)
size = ICONVCHECKSIZ;
// UTF-16 with normal prefix ?
if (size >= 2 && (!memcmp(text, "\xfe\xff", 2) ||
!memcmp(text, "\xff\xfe", 2)))
return "UTF-16";
// If we find a zero at an appropriate position, guess it's UTF-16
// anyway. This is a quite expensive test for other texts as we'll
// have to scan the whole thing.
for (i = 0; i < size - 1; i += 2) {
if (text[i] == 0 && text[i + 1] != 0)
return "UTF-16BE";
if (text[i + 1] == 0 && text[i] != 0)
return "UTF-16LE";
}
// Look for iso-2022 (rfc1468) specific escape sequences. As
// iso-2022 begins in ascii, and typically soon escapes, these
// succeed fast for a japanese text, but are quite expensive for
// any other
for (i = 0; i < size - 3; i++) {
if (text[i] == 0x1b) {
i++;
if (text[i] == '(' && strchr("BJHI", text[i + 1]))
return "ISO-2022-JP";
if (text[i] == '$' && strchr("@B(", text[i + 1]))
return "ISO-2022-JP";
}
}
// Try conversions from ascii and utf-8. These are unlikely to succeed
// by mistake.
if (transcodeErrCnt(text, size, "US-ASCII", "UTF-16BE") < 1)
return "US-ASCII";
if (transcodeErrCnt(text, size, "UTF-8", "UTF-16BE") < 1)
return "UTF-8";
hypo = 0;
for (i = 0; i < size; i++) {
if (text[i] == 0xd) {
cr = true;
break;
}
}
if (cr) {
if ((miss = transcodeErrCnt(text, size, "Shift_JIS", "EUC-JP")) < 1)
return "Shift_JIS";
if (!hypo && miss / (double)size <= ICONVALLWRAT)
hypo = "Shift_JIS";
if ((miss = transcodeErrCnt(text, size, "EUC-JP", "UTF-16BE")) < 1)
return "EUC-JP";
if (!hypo && miss / (double)size <= ICONVALLWRAT)
hypo = "EUC-JP";
} else {
if ((miss = transcodeErrCnt(text, size, "EUC-JP", "UTF-16BE")) < 1)
return "EUC-JP";
if (!hypo && miss / (double)size <= ICONVALLWRAT)
hypo = "EUC-JP";
if ((miss = transcodeErrCnt(text, size, "Shift_JIS", "EUC-JP")) < 1)
return "Shift_JIS";
if (!hypo && miss / (double)size <= ICONVALLWRAT)
hypo = "Shift_JIS";
}
if ((miss = transcodeErrCnt(text, size, "UTF-8", "UTF-16BE")) < 1)
return "UTF-8";
if (!hypo && miss / (double)size <= ICONVALLWRAT)
hypo = "UTF-8";
if ((miss = transcodeErrCnt(text, size, "CP932", "UTF-16BE")) < 1)
return "CP932";
if (!hypo && miss / (double)size <= ICONVALLWRAT)
hypo = "CP932";
return hypo ? hypo : dflt;
}
#else
#include <errno.h>
#include <cstdlib>
#include <string>
#include <iostream>
using namespace std;
#include "readfile.h"
#include "csguess.h"
int main(int argc, char **argv)
{
if (argc != 2) {
cerr << "Usage: trcsguess <filename> <default>" << endl;
exit(1);
}
const string filename = argv[1];
const string dflt = argv[2];
string text;
if (!file_to_string(filename, text)) {
cerr << "Couldnt read file, errno " << errno << endl;
exit(1);
}
cout << csguess(text, dflt) << endl;
exit(0);
}
#endif

View file

@ -1,28 +0,0 @@
/* Copyright (C) 2004 J.F.Dockes
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the
* Free Software Foundation, Inc.,
* 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
*/
#ifndef _CSGUESS_H_INCLUDED_
#define _CSGUESS_H_INCLUDED_
#include <string>
// Try to guess the character set. This might guess unicode encodings, and
// some asian charsets, but has no chance, for example, of discriminating
// betweeen the different iso8859-xx charsets.
extern std::string csguess(const std::string &in, const std::string &dflt);
#endif /* _CSGUESS_H_INCLUDED_ */

View file

@ -39,7 +39,6 @@
#include "readfile.h"
#include "indexer.h"
#include "fsindexer.h"
#include "csguess.h"
#include "transcode.h"
#include "debuglog.h"
#include "internfile.h"

View file

@ -18,7 +18,6 @@
#include "cstr.h"
#include "mimehandler.h"
#include "debuglog.h"
#include "csguess.h"
#include "readfile.h"
#include "transcode.h"
#include "mimeparse.h"

View file

@ -29,7 +29,6 @@ using namespace std;
#include "cstr.h"
#include "mh_text.h"
#include "csguess.h"
#include "debuglog.h"
#include "readfile.h"
#include "md5.h"

View file

@ -6,15 +6,15 @@ LIBS = librcl.a
all: $(LIBS)
OBJS = rclaspell.o beaglequeuecache.o cstr.o rclconfig.o rclinit.o textsplit.o unacpp.o beaglequeue.o csguess.o fsindexer.o indexer.o mimetype.o subtreelist.o htmlparse.o myhtmlparse.o mimehandler.o internfile.o mh_exec.o mh_execm.o mh_html.o mh_mail.o mh_mbox.o mh_text.o txtdcode.o docseq.o docseqdb.o docseqhist.o filtseq.o dynconf.o plaintorich.o recollq.o reslistpager.o sortseq.o wasastringtoquery.o wasatorcl.o rcldb.o rcldoc.o rclquery.o searchdata.o stemdb.o stoplist.o base64.o circache.o closefrom.o conftree.o copyfile.o debuglog.o ecrontab.o execmd.o fstreewalk.o idfile.o fileudi.o md5.o mimeparse.o netcon.o pathut.o pxattr.o rclionice.o readfile.o smallut.o transcode.o wipedir.o x11mon.o mime-getpart.o mime-parsefull.o mime-parseonlyheader.o mime-printbody.o mime-printdoc.o mime-printheader.o mime.o convert.o iodevice.o iofactory.o
DEPS = rclaspell.dep.stamp beaglequeuecache.dep.stamp cstr.dep.stamp rclconfig.dep.stamp rclinit.dep.stamp textsplit.dep.stamp unacpp.dep.stamp beaglequeue.dep.stamp csguess.dep.stamp fsindexer.dep.stamp indexer.dep.stamp mimetype.dep.stamp subtreelist.dep.stamp htmlparse.dep.stamp myhtmlparse.dep.stamp mimehandler.dep.stamp internfile.dep.stamp mh_exec.dep.stamp mh_execm.dep.stamp mh_html.dep.stamp mh_mail.dep.stamp mh_mbox.dep.stamp mh_text.dep.stamp txtdcode.dep.stamp docseq.dep.stamp docseqdb.dep.stamp docseqhist.dep.stamp filtseq.dep.stamp dynconf.dep.stamp plaintorich.dep.stamp recollq.dep.stamp reslistpager.dep.stamp sortseq.dep.stamp wasastringtoquery.dep.stamp wasatorcl.dep.stamp rcldb.dep.stamp rcldoc.dep.stamp rclquery.dep.stamp searchdata.dep.stamp stemdb.dep.stamp stoplist.dep.stamp base64.dep.stamp circache.dep.stamp closefrom.dep.stamp conftree.dep.stamp copyfile.dep.stamp debuglog.dep.stamp ecrontab.dep.stamp execmd.dep.stamp fstreewalk.dep.stamp idfile.dep.stamp fileudi.dep.stamp md5.dep.stamp mimeparse.dep.stamp netcon.dep.stamp pathut.dep.stamp pxattr.dep.stamp rclionice.dep.stamp readfile.dep.stamp smallut.dep.stamp transcode.dep.stamp wipedir.dep.stamp x11mon.dep.stamp mime-getpart.dep.stamp mime-parsefull.dep.stamp mime-parseonlyheader.dep.stamp mime-printbody.dep.stamp mime-printdoc.dep.stamp mime-printheader.dep.stamp mime.dep.stamp convert.dep.stamp iodevice.dep.stamp iofactory.dep.stamp
OBJS = rclaspell.o beaglequeuecache.o cstr.o rclconfig.o rclinit.o textsplit.o unacpp.o beaglequeue.o fsindexer.o indexer.o mimetype.o subtreelist.o htmlparse.o myhtmlparse.o mimehandler.o internfile.o mh_exec.o mh_execm.o mh_html.o mh_mail.o mh_mbox.o mh_text.o txtdcode.o docseq.o docseqdb.o docseqhist.o filtseq.o dynconf.o plaintorich.o recollq.o reslistpager.o sortseq.o wasastringtoquery.o wasatorcl.o rcldb.o rcldoc.o rclquery.o searchdata.o stemdb.o stoplist.o base64.o circache.o closefrom.o conftree.o copyfile.o debuglog.o ecrontab.o execmd.o fstreewalk.o idfile.o fileudi.o md5.o mimeparse.o netcon.o pathut.o pxattr.o rclionice.o readfile.o smallut.o transcode.o wipedir.o x11mon.o mime-getpart.o mime-parsefull.o mime-parseonlyheader.o mime-printbody.o mime-printdoc.o mime-printheader.o mime.o convert.o iodevice.o iofactory.o
DEPS = rclaspell.dep.stamp beaglequeuecache.dep.stamp cstr.dep.stamp rclconfig.dep.stamp rclinit.dep.stamp textsplit.dep.stamp unacpp.dep.stamp beaglequeue.dep.stamp fsindexer.dep.stamp indexer.dep.stamp mimetype.dep.stamp subtreelist.dep.stamp htmlparse.dep.stamp myhtmlparse.dep.stamp mimehandler.dep.stamp internfile.dep.stamp mh_exec.dep.stamp mh_execm.dep.stamp mh_html.dep.stamp mh_mail.dep.stamp mh_mbox.dep.stamp mh_text.dep.stamp txtdcode.dep.stamp docseq.dep.stamp docseqdb.dep.stamp docseqhist.dep.stamp filtseq.dep.stamp dynconf.dep.stamp plaintorich.dep.stamp recollq.dep.stamp reslistpager.dep.stamp sortseq.dep.stamp wasastringtoquery.dep.stamp wasatorcl.dep.stamp rcldb.dep.stamp rcldoc.dep.stamp rclquery.dep.stamp searchdata.dep.stamp stemdb.dep.stamp stoplist.dep.stamp base64.dep.stamp circache.dep.stamp closefrom.dep.stamp conftree.dep.stamp copyfile.dep.stamp debuglog.dep.stamp ecrontab.dep.stamp execmd.dep.stamp fstreewalk.dep.stamp idfile.dep.stamp fileudi.dep.stamp md5.dep.stamp mimeparse.dep.stamp netcon.dep.stamp pathut.dep.stamp pxattr.dep.stamp rclionice.dep.stamp readfile.dep.stamp smallut.dep.stamp transcode.dep.stamp wipedir.dep.stamp x11mon.dep.stamp mime-getpart.dep.stamp mime-parsefull.dep.stamp mime-parseonlyheader.dep.stamp mime-printbody.dep.stamp mime-printdoc.dep.stamp mime-printheader.dep.stamp mime.dep.stamp convert.dep.stamp iodevice.dep.stamp iofactory.dep.stamp
librcl.a : $(DEPS) $(OBJS) unac.o
ar ru librcl.a $(OBJS) unac.o
$(RANLIB) librcl.a
unac.o : $(depth)/unac/unac.c $(depth)/unac/unac.h $(depth)/mk/localdefs
$(CC) $(ALL_CXXFLAGS) -c $(depth)/unac/unac.c
$(CXX) $(ALL_CXXFLAGS) -c $(depth)/unac/unac.c
rclaspell.o : ../aspell/rclaspell.cpp $(depth)/mk/localdefs
$(CXX) $(ALL_CXXFLAGS) -c ../aspell/rclaspell.cpp
beaglequeuecache.o : ../common/beaglequeuecache.cpp $(depth)/mk/localdefs
@ -31,8 +31,6 @@ unacpp.o : ../common/unacpp.cpp $(depth)/mk/localdefs
$(CXX) $(ALL_CXXFLAGS) -c ../common/unacpp.cpp
beaglequeue.o : ../index/beaglequeue.cpp $(depth)/mk/localdefs
$(CXX) $(ALL_CXXFLAGS) -c ../index/beaglequeue.cpp
csguess.o : ../index/csguess.cpp $(depth)/mk/localdefs
$(CXX) $(ALL_CXXFLAGS) -c ../index/csguess.cpp
fsindexer.o : ../index/fsindexer.cpp $(depth)/mk/localdefs
$(CXX) $(ALL_CXXFLAGS) -c ../index/fsindexer.cpp
indexer.o : ../index/indexer.cpp $(depth)/mk/localdefs
@ -191,9 +189,6 @@ unacpp.dep.stamp : ../common/unacpp.cpp $(depth)/mk/localdefs
beaglequeue.dep.stamp : ../index/beaglequeue.cpp $(depth)/mk/localdefs
$(CXX) -M $(ALL_CXXFLAGS) ../index/beaglequeue.cpp > beaglequeue.dep
touch beaglequeue.dep.stamp
csguess.dep.stamp : ../index/csguess.cpp $(depth)/mk/localdefs
$(CXX) -M $(ALL_CXXFLAGS) ../index/csguess.cpp > csguess.dep
touch csguess.dep.stamp
fsindexer.dep.stamp : ../index/fsindexer.cpp $(depth)/mk/localdefs
$(CXX) -M $(ALL_CXXFLAGS) ../index/fsindexer.cpp > fsindexer.dep
touch fsindexer.dep.stamp
@ -364,7 +359,6 @@ include rclinit.dep
include textsplit.dep
include unacpp.dep
include beaglequeue.dep
include csguess.dep
include fsindexer.dep
include indexer.dep
include mimetype.dep

View file

@ -12,7 +12,6 @@ ${depth}/common/rclinit.cpp \
${depth}/common/textsplit.cpp \
${depth}/common/unacpp.cpp \
${depth}/index/beaglequeue.cpp \
${depth}/index/csguess.cpp \
${depth}/index/fsindexer.cpp \
${depth}/index/indexer.cpp \
${depth}/index/mimetype.cpp \
@ -122,7 +121,7 @@ librcl.a : \$(DEPS) \$(OBJS) unac.o
\$(RANLIB) librcl.a
unac.o : \$(depth)/unac/unac.c \$(depth)/unac/unac.h $defs
\$(CC) \$(ALL_CXXFLAGS) -c \$(depth)/unac/unac.c
\$(CXX) \$(ALL_CXXFLAGS) -c \$(depth)/unac/unac.c
EOF
for c in $SRC_CPP;do

View file

@ -142,6 +142,7 @@ filters/rclabw
filters/rclaptosidman
filters/rclaudio
filters/rclchm
filters/rcldia
filters/rcldjvu
filters/rcldoc
filters/rcldvi
@ -185,8 +186,6 @@ index/
index/Makefile
index/beaglequeue.cpp
index/beaglequeue.h
index/csguess.cpp
index/csguess.h
index/fsindexer.cpp
index/fsindexer.h
index/indexer.cpp

View file

@ -3,14 +3,21 @@ import os
import sys
sysname = os.uname()[0]
if sysname == 'Linux':
libs = ['rcl', 'xapian']
else:
libs = ['rcl', 'xapian', 'iconv']
top = os.path.join('..', '..')
libiconv = "@LIBICONV@"
if libiconv.find("-liconv") != -1:
libs = ['rcl', 'xapian', 'iconv']
else:
libs = ['rcl', 'xapian']
libxapiandir="@LIBXAPIANDIR@"
if libxapiandir != "":
libdirs = [os.path.join(top, 'lib'), libxapiandir, '/usr/local/lib']
else:
libdirs = [os.path.join(top, 'lib'), '/usr/local/lib']
# Verify that the Recoll library was compiled with the PIC options
localdefs = os.path.join(top, 'mk', 'localdefs')
try:
@ -42,7 +49,7 @@ module1 = Extension('recoll',
os.path.join(top, 'unac')
],
libraries = libs,
library_dirs = [os.path.join(top, 'lib'), '/usr/local/lib'],
library_dirs = libdirs,
sources = ['pyrecoll.cpp',
])

View file

@ -176,7 +176,7 @@
</message>
<message>
<source>Filter sizes</source>
<translation>Tamaños de filtros</translation>
<translation>Filtro de tamaños</translation>
</message>
</context>
<context>

View file

@ -88,12 +88,6 @@ unix {
UNAME = $$system(uname -s)
contains( UNAME, [lL]inux ) {
LIBS -= -liconv
LIBS += -ldl -lX11
}
contains( UNAME, GNU/kFreeBSD ) {
LIBS -= -liconv
LIBS += -ldl -lX11
}

View file

@ -231,9 +231,16 @@ string QtGuiResListPager::iconUrl(RclConfig *config, Rcl::Doc& doc)
ConfIndexer::docsToPaths(docs, paths);
if (!paths.empty()) {
string path;
LOGDEB2(("ResList::iconUrl: source path [%s]\n", paths[0].c_str()));
if (thumbPathForUrl(cstr_fileu + paths[0], 128, path)) {
LOGDEB2(("ResList::iconUrl: icon path [%s]\n", path.c_str()));
return cstr_fileu + path;
} else {
LOGDEB2(("ResList::iconUrl: no icon: path [%s]\n",
path.c_str()));
}
} else {
LOGDEB(("ResList::iconUrl: docsToPaths failed\n"));
}
}
return ResListPager::iconUrl(config, doc);

View file

@ -71,6 +71,7 @@ application/vnd.wordperfect = exec wpd2html;mimetype=text/html
application/x-abiword = exec rclabw
application/x-awk = internal text/plain
application/x-chm = execm rclchm
application/x-dia-diagram = execm rcldia;mimetype=text/html;charset=utf-8
application/x-dvi = exec rcldvi
application/x-flac = execm rclaudio
application/x-gnuinfo = execm rclinfo
@ -107,6 +108,7 @@ text/x-fictionbook = exec rclfb2
text/x-gaim-log = exec rclgaim
text/x-html-sidux-man = exec rclsiduxman
text/x-html-aptosid-man = exec rclaptosidman
text/x-chm-html = internal text/html
text/x-ini = internal text/plain
text/x-mail = internal
text/x-man = exec rclman
@ -153,6 +155,7 @@ application/vnd.sun.xml.writer.global = wordprocessing
application/vnd.sun.xml.writer.template = wordprocessing
application/vnd.wordperfect = wordprocessing
application/x-abiword = wordprocessing
application/x-dia-diagram = drawing
application/x-dvi = document
application/x-flac = sownd
application/x-fsdirectory = folder
@ -283,6 +286,7 @@ message = message/rfc822 \
other = application/vnd.sun.xml.draw \
application/vnd.sun.xml.draw.template \
application/vnd.sun.xml.math \
application/x-dia-diagram \
application/x-fsdirectory \
application/x-mimehtml \
application/x-rar \

View file

@ -41,6 +41,7 @@
.djvu = image/vnd.djvu
.svg = image/svg+xml
.dia = application/x-dia-diagram
.gz = application/x-gzip
.Z = application/x-gzip

View file

@ -52,6 +52,7 @@ application/vnd.sun.xml.writer.template = libreoffice %f
application/vnd.wordperfect = libreoffice %f
application/x-chm = kchmviewer %f
application/x-dia-diagram = dia %f
application/x-fsdirectory = dolphin %f
application/x-gnuinfo = xterm -e "info -f %f"
application/x-gnumeric = gnumeric %f
@ -82,6 +83,7 @@ text/x-c+ = emacsclient %f
text/x-c++ = emacsclient %f
text/x-html-sidux-man = konqueror %f
text/x-html-aptosid-man = iceweasel %f
text/x-chm-html = openchm %f %i
text/x-ini = emacsclient %f
text/x-man = xterm -u8 -e "groff -T ascii -man %f | more"
text/x-python = idle %f

View file

@ -65,6 +65,17 @@ indexstemminglanguages = english
# match your LANG and is not 8859-1, set it here.
# defaultcharset = iso-8859-1
# A list of characters, encoded in UTF-8, which should be handled specially
# when converting text to unaccented lowercase. For example, in Swedish,
# the letter a with diaeresis has full alphabet citizenship and should not
# be turned into an a.
# Each element in the space-separated list has the special character as
# first element and the translation following. The handling of both the
# lowercase and upper-case versions of a character should be specified, as
# appartenance to the list will turn-off both standard accent and case
# processing. Example for Swedish:
# unac_except_trans = åå Åå ää Ää öö Öö
# Where to store the database (directory). This may be an absolute path,
# else it is taken as relative to the configuration directory (-c argument
# or $RECOLL_CONFDIR).

View file

@ -17,15 +17,57 @@
*/
#ifdef HAVE_CONFIG_H
#ifdef RECOLL_DATADIR
#include "autoconfig.h"
#else
#include "config.h"
#endif /* RECOLL */
#endif /* HAVE_CONFIG_H */
#ifdef RECOLL_DATADIR
/* Yes, recoll unac is actually c++, lets face modernity, I will not be
caught writing another binary search */
#include <vector>
#include <map>
#include <string>
#include <algorithm>
using std::string;
using std::vector;
using std::map;
#include "smallut.h"
/*
Storage for the exception translations. These are chars which
should not be translated according to what UnicodeData says, but
instead according to some local rule. There will usually be very
few of them, but they must be looked up for every translated char.
We use a sorted vector for fastest elimination by binary search and
a vector<string> to store the translations
*/
static vector<unsigned short> except_chars;
static vector<string> except_trans;
static inline size_t is_except_char(unsigned short c)
{
vector<unsigned short>::iterator it =
std::lower_bound(except_chars.begin(), except_chars.end(), c);
if (it == except_chars.end() || *it != c) {
return (size_t(-1));
}
return std::distance(except_chars.begin(), it);
}
#endif /* RECOLL_DATADIR */
/*
* If configure.in has not defined this symbol, assume const. It
* does not harm much: a warning will be issued during compilation.
*/
#ifndef ICONV_CONST
#ifdef RCL_ICONV_INBUF_CONST
#define ICONV_CONST const
#else
#define ICONV_CONST
#endif
#endif /* ICONV_CONST */
#include <stdlib.h>
@ -12622,12 +12664,12 @@ int unacmaybefold_string_utf16(const char* in, size_t in_length,
char* out;
int out_size;
int out_length;
int i;
unsigned int i;
out_size = in_length > 0 ? in_length : 1024;
out = *outp;
out = realloc(out, out_size + 1);
out = (char*)realloc(out, out_size + 1);
if(out == 0) {
if(debug_level >= UNAC_DEBUG_LOW)
DEBUG("realloc %d bytes failed\n", out_size+1);
@ -12646,11 +12688,25 @@ int unacmaybefold_string_utf16(const char* in, size_t in_length,
/*
* Lookup the tables for decomposition information
*/
#ifdef RECOLL_DATADIR
size_t idx;
if (except_chars.size() != 0 && (idx=is_except_char(c)) != (size_t)-1) {
p = (unsigned short *)(except_trans[idx].c_str() + 2);
l = (except_trans[idx].size() - 2) / 2;
/* unsigned char *cp = (unsigned char *)p;
fprintf(stderr, "l %d cp[0] %x cp[1] %x\n", l, (unsigned int)cp[0],
(unsigned int)cp[1]);*/
} else {
#endif /* RECOLL_DATADIR */
if (dofold) {
unacfold_char_utf16(c, p, l);
} else {
unac_char_utf16(c, p, l);
}
#ifdef RECOLL_DATADIR
}
#endif /* RECOLL_DATADIR */
/*
* Explain what's done in great detail
*/
@ -12678,7 +12734,7 @@ int unacmaybefold_string_utf16(const char* in, size_t in_length,
char *saved;
out_size += ((l + 1) * 2) + 1024;
saved = out;
out = realloc(out, out_size);
out = (char *)realloc(out, out_size);
if(out == 0) {
if(debug_level >= UNAC_DEBUG_LOW)
DEBUG("realloc %d bytes failed\n", out_size);
@ -12798,7 +12854,7 @@ static int convert(const char* from, const char* to,
out_size = in_length > 0 ? in_length : 1024;
out = *outp;
out = realloc(out, out_size + 1);
out = (char *)realloc(out, out_size + 1);
if(out == 0) {
/* *outp still valid, no freeing */
if(debug_level >= UNAC_DEBUG_LOW)
@ -12884,7 +12940,7 @@ static int convert(const char* from, const char* to,
{
char *saved = out_base;
/* +1 for null */
out_base = realloc(out_base, out_size + 1);
out_base = (char *)realloc(out_base, out_size + 1);
if (out_base == 0) {
/* *outp potentially not valid any more. Free here,
* and zero out */
@ -12929,7 +12985,7 @@ int unacmaybefold_string(const char* charset,
*/
if (in_length <= 0) {
if(!*outp) {
if ((*outp = malloc(32)) == 0)
if ((*outp = (char*)malloc(32)) == 0)
return -1;
}
(*outp)[0] = '\0';
@ -12975,3 +13031,64 @@ const char* unac_version(void)
return UNAC_VERSION;
}
#ifdef RECOLL_DATADIR
void unac_set_except_translations(const char *spectrans)
{
except_chars.clear();
except_trans.clear();
if (!spectrans || !spectrans[0])
return;
// The translation tables out of Unicode are in machine byte order (we
// just let the compiler read the values).
// For the translation part, we need to choose our encoding in accordance )
// (16BE or 16LE depending on processor)
// On the contrary, the source char is always to be compared to
// the input text, which is encoded in UTF-16BE ... What a mess.
static const char *machinecoding = 0;
bool littleendian = true;
if (machinecoding == 0) {
const char* charshort = "\001\002";
short *ip = (short *)charshort;
if (*ip == 0x0102) {
littleendian = false;
machinecoding = "UTF-16BE";
} else {
littleendian = true;
machinecoding = "UTF-16LE";
}
}
vector<string> vtrans;
stringToStrings(spectrans, vtrans);
for (vector<string>::iterator it = vtrans.begin();
it != vtrans.end(); it++) {
/* Convert the whole thing to utf-16be/le according to endianness */
char *out = 0;
size_t outsize;
if (convert("UTF-8", machinecoding,
it->c_str(), it->size(),
&out, &outsize) != 0 || outsize < 2)
continue;
/* The source char must be utf-16be as this is what we convert the
input text to for internal processing */
unsigned short ch;
if (littleendian)
ch = (out[1] << 8) | (out[0] & 0xff);
else
ch = (out[0] << 8) | (out[1] & 0xff);
/* fprintf(stderr, "outsize %d Ch is 0x%hx\n", int(outsize), ch);*/
except_chars.push_back(ch);
// We keep ch as the first 2 bytes in the translation so that
// both vectors sort identically
except_trans.push_back(string((const char *)out, outsize));
free(out);
}
std::sort(except_chars.begin(), except_chars.end());
std::sort(except_trans.begin(), except_trans.end());
}
#endif /* RECOLL_DATADIR */

View file

@ -116,6 +116,22 @@ int unacfold_string(const char* charset,
/* To be called before starting threads in mt programs */
void unac_init_mt();
#ifdef RECOLL_DATADIR
#include <string>
/**
* Set exceptions for unaccenting, for characters which should not be
* handled according to what the Unicode tables say. For example "a
* with circle above" should not be stripped to a in swedish, etc.
*
* @param spectrans defines the translations as a blank separated list of
* UTF-8 strings. Inside each string, the first character is the exception
* the rest is the translation (which may be empty). You can use double
* quotes for translations which should include white space. The double-quote
* can't be an exception character, deal with it...
*/
void unac_set_except_translations(const char *spectrans);
#endif /* RECOLL_DATADIR */
/*
* Return unac version number.
*/

View file

@ -389,6 +389,52 @@ bool path_isdir(const string& path)
return false;
}
// Allowed punctuation in the path part of an URI according to RFC2396
// -_.!~*'():@&=+$,
/*
21 !
22 "
23 #
24 $
25 %
26 &
27 '
28 (
29 )
2A *
2B +
2C ,
2D -
2E .
2F /
30 0
...
39 9
3A :
3B ;
3C <
3D =
3E >
3F ?
40 @
41 A
...
5A Z
5B [
5C \
5D ]
5E ^
5F _
60 `
61 a
...
7A z
7B {
7C |
7D }
7E ~
7F DEL
*/
string url_encode(const string& url, string::size_type offs)
{
string out = url.substr(0, offs);
@ -397,24 +443,23 @@ string url_encode(const string& url, string::size_type offs)
int c;
const char *h = "0123456789ABCDEF";
c = cp[i];
if(c <= 0x1f ||
if (c <= 0x20 ||
c >= 0x7f ||
c == '<' ||
c == '>' ||
c == ' ' ||
c == '\t'||
c == '"' ||
c == '#' ||
c == '%' ||
c == '{' ||
c == '}' ||
c == '|' ||
c == '\\' ||
c == '^' ||
c == '~'||
c == ';' ||
c == '<' ||
c == '>' ||
c == '?' ||
c == '[' ||
c == '\\' ||
c == ']' ||
c == '`') {
c == '^' ||
c == '`' ||
c == '{' ||
c == '|' ||
c == '}' ) {
out += '%';
out += h[(c >> 4) & 0xf];
out += h[c & 0xf];
@ -445,7 +490,9 @@ string url_gpath(const string& url)
return path_canon(url.substr(colon+1));
}
// Convert to file path if url is like file://
// Convert to file path if url is like file:
// Note: this only works with our internal pseudo-urls which are not
// encoded/escaped
string fileurltolocalpath(string url)
{
if (url.find("file://") == 0)
@ -570,7 +617,8 @@ static const string thmbdirnormal = ".thumbnails/normal";
static void thumbname(const string& url, string& name)
{
string digest;
MD5String(url, digest);
string l_url = url_encode(url);
MD5String(l_url, digest);
MD5HexPrint(digest, name);
name += ".png";
}

View file

@ -15,6 +15,7 @@
* 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
*/
#include <stdio.h>
#include <unistd.h>
#include "rclionice.h"
#include "execmd.h"

View file

@ -15,6 +15,8 @@ daemSkippedPaths = \
defaultcharset = iso-8859-1
unac_except_trans = åå Åå ää Ää öö Öö
[/home/dockes/projets/fulltext/testrecoll/special]
defaultcharset = iso-8859-1
[/home/dockes/projets/fulltext/testrecoll/txt]

14
tests/dia/dia.sh Executable file
View file

@ -0,0 +1,14 @@
#!/bin/sh
topdir=`dirname $0`/..
. $topdir/shared.sh
initvariables $0
(
recollq DEPARTMENT NumberOfEmployees
recollq Within5Min WaterLevel KeptBelow
) 2> $mystderr | egrep -v '^Recoll query: ' > $mystdout
diff -w ${myname}.txt $mystdout > $mydiffs 2>&1
checkresult

4
tests/dia/dia.txt Normal file
View file

@ -0,0 +1,4 @@
1 results
application/x-dia-diagram [file:///home/dockes/projets/fulltext/testrecoll/dia/ER-demo.dia] [ER-demo.dia] 60803 bytes
1 results
application/x-dia-diagram [file:///home/dockes/projets/fulltext/testrecoll/dia/Istar.dia] [Istar.dia] 3537 bytes

View file

@ -8,7 +8,9 @@ initvariables $0
(
recollq 'Bienvenue Dans Univers De Lyx'
recollq 'Welcome To Lyx'
recollq 'Udvozli Ont A LyX'
recollq 'LyX rendkivul jol dokumentalt'
# cant use this one because o-diaeresis is in the swedish unacex
# recollq 'Udvozli Ont A LyX'
) 2> $mystderr | egrep -v '^Recoll query: ' > $mystdout
diff -w ${myname}.txt $mystdout > $mydiffs 2>&1

20
tests/unacex/unacex.sh Executable file
View file

@ -0,0 +1,20 @@
#!/bin/sh
topdir=`dirname $0`/..
. $topdir/shared.sh
initvariables $0
# We need an utf-8 locale for the 1st command to properly read its argument
export LANG=fr_FR.UTF-8
(
# Should succeed
recollq '"strippes: UNACEXååääöö"'
# Should fail
recollq '"strippes: UNACEXaaaaoo"'
) 2> $mystderr | egrep -v '^Recoll query: ' > $mystdout
diff -w ${myname}.txt $mystdout > $mydiffs 2>&1
checkresult

3
tests/unacex/unacex.txt Normal file
View file

@ -0,0 +1,3 @@
1 results
text/html [file:///home/dockes/projets/fulltext/testrecoll/unacex/swedishchars.html] [Some chars] 293 bytes
0 results

View file

@ -17,15 +17,57 @@
*/
#ifdef HAVE_CONFIG_H
#ifdef RECOLL_DATADIR
#include "autoconfig.h"
#else
#include "config.h"
#endif /* RECOLL */
#endif /* HAVE_CONFIG_H */
#ifdef RECOLL_DATADIR
/* Yes, recoll unac is actually c++, lets face modernity, I will not be
caught writing another binary search */
#include <vector>
#include <map>
#include <string>
#include <algorithm>
using std::string;
using std::vector;
using std::map;
#include "smallut.h"
/*
Storage for the exception translations. These are chars which
should not be translated according to what UnicodeData says, but
instead according to some local rule. There will usually be very
few of them, but they must be looked up for every translated char.
We use a sorted vector for fastest elimination by binary search and
a vector<string> to store the translations
*/
static vector<unsigned short> except_chars;
static vector<string> except_trans;
static inline size_t is_except_char(unsigned short c)
{
vector<unsigned short>::iterator it =
std::lower_bound(except_chars.begin(), except_chars.end(), c);
if (it == except_chars.end() || *it != c) {
return (size_t(-1));
}
return std::distance(except_chars.begin(), it);
}
#endif /* RECOLL_DATADIR */
/*
* If configure.in has not defined this symbol, assume const. It
* does not harm much: a warning will be issued during compilation.
*/
#ifndef ICONV_CONST
#ifdef RCL_ICONV_INBUF_CONST
#define ICONV_CONST const
#else
#define ICONV_CONST
#endif
#endif /* ICONV_CONST */
#include <stdlib.h>
@ -12622,12 +12664,12 @@ int unacmaybefold_string_utf16(const char* in, size_t in_length,
char* out;
int out_size;
int out_length;
int i;
unsigned int i;
out_size = in_length > 0 ? in_length : 1024;
out = *outp;
out = realloc(out, out_size + 1);
out = (char*)realloc(out, out_size + 1);
if(out == 0) {
if(debug_level >= UNAC_DEBUG_LOW)
DEBUG("realloc %d bytes failed\n", out_size+1);
@ -12646,11 +12688,25 @@ int unacmaybefold_string_utf16(const char* in, size_t in_length,
/*
* Lookup the tables for decomposition information
*/
#ifdef RECOLL_DATADIR
size_t idx;
if (except_chars.size() != 0 && (idx=is_except_char(c)) != (size_t)-1) {
p = (unsigned short *)(except_trans[idx].c_str() + 2);
l = (except_trans[idx].size() - 2) / 2;
/* unsigned char *cp = (unsigned char *)p;
fprintf(stderr, "l %d cp[0] %x cp[1] %x\n", l, (unsigned int)cp[0],
(unsigned int)cp[1]);*/
} else {
#endif /* RECOLL_DATADIR */
if (dofold) {
unacfold_char_utf16(c, p, l);
} else {
unac_char_utf16(c, p, l);
}
#ifdef RECOLL_DATADIR
}
#endif /* RECOLL_DATADIR */
/*
* Explain what's done in great detail
*/
@ -12678,7 +12734,7 @@ int unacmaybefold_string_utf16(const char* in, size_t in_length,
char *saved;
out_size += ((l + 1) * 2) + 1024;
saved = out;
out = realloc(out, out_size);
out = (char *)realloc(out, out_size);
if(out == 0) {
if(debug_level >= UNAC_DEBUG_LOW)
DEBUG("realloc %d bytes failed\n", out_size);
@ -12798,7 +12854,7 @@ static int convert(const char* from, const char* to,
out_size = in_length > 0 ? in_length : 1024;
out = *outp;
out = realloc(out, out_size + 1);
out = (char *)realloc(out, out_size + 1);
if(out == 0) {
/* *outp still valid, no freeing */
if(debug_level >= UNAC_DEBUG_LOW)
@ -12884,7 +12940,7 @@ static int convert(const char* from, const char* to,
{
char *saved = out_base;
/* +1 for null */
out_base = realloc(out_base, out_size + 1);
out_base = (char *)realloc(out_base, out_size + 1);
if (out_base == 0) {
/* *outp potentially not valid any more. Free here,
* and zero out */
@ -12929,7 +12985,7 @@ int unacmaybefold_string(const char* charset,
*/
if (in_length <= 0) {
if(!*outp) {
if ((*outp = malloc(32)) == 0)
if ((*outp = (char*)malloc(32)) == 0)
return -1;
}
(*outp)[0] = '\0';
@ -12975,3 +13031,64 @@ const char* unac_version(void)
return UNAC_VERSION;
}
#ifdef RECOLL_DATADIR
void unac_set_except_translations(const char *spectrans)
{
except_chars.clear();
except_trans.clear();
if (!spectrans || !spectrans[0])
return;
// The translation tables out of Unicode are in machine byte order (we
// just let the compiler read the values).
// For the translation part, we need to choose our encoding in accordance )
// (16BE or 16LE depending on processor)
// On the contrary, the source char is always to be compared to
// the input text, which is encoded in UTF-16BE ... What a mess.
static const char *machinecoding = 0;
bool littleendian = true;
if (machinecoding == 0) {
const char* charshort = "\001\002";
short *ip = (short *)charshort;
if (*ip == 0x0102) {
littleendian = false;
machinecoding = "UTF-16BE";
} else {
littleendian = true;
machinecoding = "UTF-16LE";
}
}
vector<string> vtrans;
stringToStrings(spectrans, vtrans);
for (vector<string>::iterator it = vtrans.begin();
it != vtrans.end(); it++) {
/* Convert the whole thing to utf-16be/le according to endianness */
char *out = 0;
size_t outsize;
if (convert("UTF-8", machinecoding,
it->c_str(), it->size(),
&out, &outsize) != 0 || outsize < 2)
continue;
/* The source char must be utf-16be as this is what we convert the
input text to for internal processing */
unsigned short ch;
if (littleendian)
ch = (out[1] << 8) | (out[0] & 0xff);
else
ch = (out[0] << 8) | (out[1] & 0xff);
/* fprintf(stderr, "outsize %d Ch is 0x%hx\n", int(outsize), ch);*/
except_chars.push_back(ch);
// We keep ch as the first 2 bytes in the translation so that
// both vectors sort identically
except_trans.push_back(string((const char *)out, outsize));
free(out);
}
std::sort(except_chars.begin(), except_chars.end());
std::sort(except_trans.begin(), except_trans.end());
}
#endif /* RECOLL_DATADIR */

View file

@ -116,6 +116,22 @@ int unacfold_string(const char* charset,
/* To be called before starting threads in mt programs */
void unac_init_mt();
#ifdef RECOLL_DATADIR
#include <string>
/**
* Set exceptions for unaccenting, for characters which should not be
* handled according to what the Unicode tables say. For example "a
* with circle above" should not be stripped to a in swedish, etc.
*
* @param spectrans defines the translations as a blank separated list of
* UTF-8 strings. Inside each string, the first character is the exception
* the rest is the translation (which may be empty). You can use double
* quotes for translations which should include white space. The double-quote
* can't be an exception character, deal with it...
*/
void unac_set_except_translations(const char *spectrans);
#endif /* RECOLL_DATADIR */
/*
* Return unac version number.
*/

View file

@ -12,7 +12,7 @@
<meta name="Keywords" content=
"full text search, desktop search, unix, linux">
<meta http-equiv="Content-language" content="en">
<meta http-equiv="content-type" content="text/html; charset=iso-8859-1">
<meta http-equiv="content-type" content="text/html; charset=utf-8">
<meta name="robots" content="All,Index,Follow">
<link type="text/css" rel="stylesheet" href="styles/style.css">
@ -34,6 +34,8 @@
<ul>
<li><a href="usermanual/index.html">English, HTML</a></li>
<li><a href="http://stupidbeauty.com/Blog/2012/03/recoll%E7%94%A8%E6%88%B6%E6%89%8B%E5%86%8A%E7%BF%BB%E8%AD%AF%EF%BC%8Crecoll-user-manual-2/">
中文HTML</a></li>
<li><a href="usermanual/recoll_user_manual.pdf">English, PDF</a></li>
<li><a href="http://mcz.altervista.org/Pagine/usermanual-italian.html">
Italian (rather old)</a></li>

View file

@ -90,8 +90,8 @@
compressed versions). Some types are handled internally (no
external application needed). Other types need a separate
application to be installed to extract the text. Types that
only need very common utilities (awk/sed/groff etc.) are
listed in the native section.</p>
only need very common utilities (awk/sed/groff/Python etc.)
are listed in the native section.</p>
<h4>File types indexed natively</h4>
@ -100,22 +100,25 @@
<li><span class="literal">html</span>.</li>
<li><span class="literal">maildir</span> and <span class=
"literal">mailbox</span> (<span class=
"literal">Mozilla</span>, <span class=
"literal">Thunderbird</span> and <span class=
"literal">Evolution</span> mail ok).</li>
<li><span class="literal">maildir</span> and
<span class="literal">mailbox</span> (
<span class="literal">Mozilla</span>,
<span class="literal">Thunderbird</span> and
<span class="literal">Evolution</span>mail ok).
</li>
<li><span class="literal">gaim</span> and <span class=
"literal">purple</span> log files.</li>
<li><span class="literal">gaim</span> and
<span class="literal">purple</span> log files.</li>
<li><span class="literal">Lyx</span> files (needs <span
class="literal">Lyx</span> to be installed).</li>
<li><span class="literal">Lyx</span> files (needs
<span class="literal">Lyx</span> to be installed).</li>
<li><span class="literal">Scribus</span> files.</li>
<li><span class="literal">Man pages</span> (need <span
class="command">groff</span>).</li>
<li><span class="literal">Man pages</span> (needs
<span class="command">groff</span>).</li>
<li><span class="literal">Dia</span> diagrams.</li>
</ul>
<h4>File types indexed with external helpers</h4>
@ -124,8 +127,9 @@
command in addition to the applications specifically listed.</p>
<h5>The XML ones</h5>
<p>The following types need <span class=
"command">xsltproc</span> from the <b>libxslt</b> package.
<p>The following types need <span class="command">
xsltproc</span> from the <b>libxslt</b> package.
Quite a few also need <span class="command">unzip</span>:</p>
<ul>
@ -148,6 +152,9 @@
<h5>Other formats</h5>
<p>The following need miscellaneous helper programs to decode
the internal formats.</p>
<ul>
<li><span class="literal">pdf</span> with the <span class=
"command">pdftotext</span> command, which can be installed
@ -166,7 +173,8 @@
"http://vitus.wagner.pp.ru/software/catdoc/">catdoc</a> utilities.</li>
<li><span class="literal">CHM (Microsoft help)</span> files
with <span class="command">Python, <a href="http://gnochm.sourceforge.net/pychm.html">pychm</a>
with <span class="command">Python,
<a href="http://gnochm.sourceforge.net/pychm.html">pychm</a>
and <a href="http://www.jedrea.com/chmlib/">chmlib</a></span>.</li>
<li><span class="literal">GNU info</span> files

View file

@ -75,6 +75,11 @@
should upgrade.</p>
</blockquote>
<h2>DIA files</h2>
<p><a href="rcldia">rcldia</a> is a new filter
for <a href="http://projects.gnome.org/dia/">Dia</a> files,
contributed by Stefan Friedel.</p>
<h2>CHM files</h2>
<p><a href="rclchm">rclchm</a>. The previous version of the
filter mishandled files which had encoded internal URLs (not

View file

@ -7,7 +7,7 @@
<meta name="Description" content="Recoll is a personal desktop full text search application for Unix, Linux and Mac OS X, based on the Xapian search engine library.">
<meta name="Keywords" content="full text search,fulltext,full text,search,desktop search,unix,linux,open source,free">
<meta http-equiv="Content-language" content="en">
<meta http-equiv="content-type" content="text/html; charset=iso-8859-1">
<meta http-equiv="content-type" content="text/html; charset=utf-8">
<meta name="robots" content="All,Index,Follow">
<link type="text/css" rel="stylesheet" href="styles/style.css">
</head>
@ -69,20 +69,19 @@
<h2>News: </h2>
<div class="news">
<ul>
<li>2012-03-27: The Unity Lens can now handle results for embedded
documents, using the Recoll GUI as a proxy to extract and
display them.</li>
<li>2012-04-07: we now have a Chinese user manual: Recoll现在有中文手册咯
<a href="http://stupidbeauty.com/Blog/2012/03/recoll%E7%94%A8%E6%88%B6%E6%89%8B%E5%86%8A%E7%BF%BB%E8%AD%AF%EF%BC%8Crecoll-user-manual-2/">
Recoll中文手册HTML</a></li>
<li>2012-03-26: Recoll gets a Ubuntu Unity Lens. If you are running
<li>2012-03-27: Recoll gets a Ubuntu Unity Lens. If you are running
an Ubuntu release where this makes sense, you can install
the recoll-lens package from the
<a href="https://launchpad.net/~recoll-backports/+archive/recoll-1.15-on">
Recoll PPA</a>. <br />
<i><b>Please note</b></i>: for now, the Lens will only return
results which are normal files, not embedded documents.
Ie: no mbox messages, no attachments, no archive
members, etc. For now, you'll have to use the full Recoll
GUI to find these.</li>
Recoll PPA</a>. The Lens uses the Recoll GUI as a proxy to
extract and display embedded documents, which native utility
can't reach directly. And of course you still need to run
the GUI (or the command line recollindex) to get indexing
going !</li>
<li>2012-03-24: Release 1.17 is out, see
the <a href="release-1.17.html">Release notes</a>. </li>

View file

@ -0,0 +1,2 @@
#!/bin/sh
onlylist=1 photalb . .

View file

@ -1,9 +1,30 @@
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<title>Photos</title>
<title>Recoll screenshots</title>
<meta name="generator" content="HTML Tidy, see www.w3.org">
<meta name="Author" content="Jean-Francois Dockes">
<meta name="Description" content=
"recoll is a simple full-text search system for unix and linux
based on the powerful and mature xapian engine">
<meta name="Keywords" content=
"full text search, desktop search, unix, linux">
<meta http-equiv="Content-language" content="en">
<meta http-equiv="content-type" content="text/html; charset=utf-8">
<meta name="robots" content="All,Index,Follow">
<link type="text/css" rel="stylesheet" href="../styles/style.css">
</head>
<body>
<div class="content">
<h1>Recoll Screenshots</h1>
<li><a href="../index.html">Back to Recoll home</a></li>
<table>
<tr>
<td align="center"><a href="recoll0.html"><img src="recoll0-thumb.png"></a></td>
@ -19,4 +40,5 @@
</tr>
<tr>
<td align="center"><a href="recoll-HTML_search_results.html"><img src="recoll-HTML_search_results-thumb.png"></a></td>
</tr></table></body></html>
</tr></table>
</body></html>

View file

@ -0,0 +1,27 @@
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<title>Recoll screenshots</title>
<meta name="generator" content="HTML Tidy, see www.w3.org">
<meta name="Author" content="Jean-Francois Dockes">
<meta name="Description" content=
"recoll is a simple full-text search system for unix and linux
based on the powerful and mature xapian engine">
<meta name="Keywords" content=
"full text search, desktop search, unix, linux">
<meta http-equiv="Content-language" content="en">
<meta http-equiv="content-type" content="text/html; charset=utf-8">
<meta name="robots" content="All,Index,Follow">
<link type="text/css" rel="stylesheet" href="../styles/style.css">
</head>
<body>
<div class="content">
<h1>Recoll Screenshots</h1>
<li><a href="../index.html">Back to Recoll home</a></li>

View file

@ -4,9 +4,9 @@
<title>Photo</title>
</head>
<body>
<p><a href="recoll_chinese.html">Prev</a> <a href="..">Up</a>
<p><a href="recoll_chinese.html">Prev</a> <a href=".">Up</a>
<a href="recoll0.html">Next</a>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
<a href="recoll-HTML_search_results.png">Raw Image</a></p>
<a href="recoll-HTML_search_results.png">Image</a></p>
<p>A customized result list, thanks to Michael Croes. The html code follows,
it should be pasted into the
<i>Preferences->Query&nbsp;Configuration->Result&nbsp;paragraph&nbsp;format&nbsp;string</i> entry.

View file

@ -4,9 +4,9 @@
<title>Photo</title>
</head>
<body>
<p><a href="..">Prev</a> <a href="..">Up</a>
<p><a href=".">Prev</a> <a href=".">Up</a>
<a href="result-table.html">Next</a>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
<a href="recoll0.png">Raw Image</a></p>
<a href="recoll0.png">Image</a></p>
<p>Search results.</p>
<p><img height="90%" src="recoll0.png"></p>
</body>

View file

@ -4,9 +4,9 @@
<title>Photo</title>
</head>
<body>
<p><a href="result-table.html">Prev</a> <a href="..">Up</a>
<p><a href="result-table.html">Prev</a> <a href=".">Up</a>
<a href="recoll2.html">Next</a>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
<a href="recoll1.png">Raw Image</a></p>
<a href="recoll1.png">Image</a></p>
<p>A result list with a preview window open.</p>
<p><img height="90%" src="recoll1.png"></p>
</body>

Binary file not shown.

Before

Width:  |  Height:  |  Size: 45 KiB

After

Width:  |  Height:  |  Size: 25 KiB

Before After
Before After

View file

@ -4,10 +4,10 @@
<title>Photo</title>
</head>
<body>
<p><a href="recoll1.html">Prev</a> <a href="..">Up</a>
<p><a href="recoll1.html">Prev</a> <a href=".">Up</a>
<a href="recoll3.html">Next</a>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
<a href="recoll2.png">Raw Image</a></p>
<p>The advanced search dialog.</p>
<a href="recoll2.png">Image</a></p>
<p>The two tabs in the advanced search dialog.</p>
<p><img height="90%" src="recoll2.png"></p>
</body>
</html>

Binary file not shown.

Before

Width:  |  Height:  |  Size: 46 KiB

After

Width:  |  Height:  |  Size: 46 KiB

Before After
Before After

View file

@ -1 +1 @@
The advanced search dialog.
The two tabs in the advanced search dialog.

View file

@ -4,9 +4,9 @@
<title>Photo</title>
</head>
<body>
<p><a href="recoll2.html">Prev</a> <a href="..">Up</a>
<p><a href="recoll2.html">Prev</a> <a href=".">Up</a>
<a href="recoll4.html">Next</a>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
<a href="recoll3.png">Raw Image</a></p>
<a href="recoll3.png">Image</a></p>
<p>A result list from which the native application (firefox)
was started by clicking the Edit link.</p>
<p><img height="90%" src="recoll3.png"></p>

View file

@ -4,9 +4,9 @@
<title>Photo</title>
</head>
<body>
<p><a href="recoll3.html">Prev</a> <a href="..">Up</a>
<p><a href="recoll3.html">Prev</a> <a href=".">Up</a>
<a href="recoll5.html">Next</a>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
<a href="recoll4.png">Raw Image</a></p>
<a href="recoll4.png">Image</a></p>
<p>The document history window looks a little like a result list
I'm afraid...</p>
<p><img height="90%" src="recoll4.png"></p>

View file

@ -4,9 +4,9 @@
<title>Photo</title>
</head>
<body>
<p><a href="recoll4.html">Prev</a> <a href="..">Up</a>
<p><a href="recoll4.html">Prev</a> <a href=".">Up</a>
<a href="recoll_chinese.html">Next</a>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
<a href="recoll5.png">Raw Image</a></p>
<a href="recoll5.png">Image</a></p>
<p>The term explorer tool in phonetic mode.</p>
<p><img height="90%" src="recoll5.png"></p>
</body>

View file

@ -4,9 +4,9 @@
<title>Photo</title>
</head>
<body>
<p><a href="recoll5.html">Prev</a> <a href="..">Up</a>
<p><a href="recoll5.html">Prev</a> <a href=".">Up</a>
<a href="recoll-HTML_search_results.html">Next</a>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
<a href="recoll_chinese.png">Raw Image</a></p>
<a href="recoll_chinese.png">Image</a></p>
<p>Recoll searching Chinese text. Chinese text search is based on n-grams and
relatively rough, but still useful (I am told).</p>
<p><img height="90%" src="recoll_chinese.png"></p>

View file

@ -4,9 +4,9 @@
<title>Photo</title>
</head>
<body>
<p><a href="recoll0.html">Prev</a> <a href="..">Up</a>
<p><a href="recoll0.html">Prev</a> <a href=".">Up</a>
<a href="recoll1.html">Next</a>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
<a href="result-table.png">Raw Image</a></p>
<a href="result-table.png">Image</a></p>
<p></p>
<p><img height="90%" src="result-table.png"></p>
</body>