Re: [translate-pootle] [Fwd: Re: Wordforge]
Gintautas Miliauskas wrote:
> Hello,
>
> I've started sketching the API, but I would like to have a chat on IRC
> with the developers who are working on the new base classes. Should I
> use your branch or start a new one? When can I find you online, guys?
> Perhaps I can find you on Jabber? Ottavio? David? Friedle?
>
Saw you on IRC over the weekend but unfortunately couldn't chat. #pootle
on irc.freenode.net is where we usually hang out
My jabber address is david at jabber.obsidian.co.za
I think the first thing is for you to look at the current base classes
and API, and then we can discuss how it works (I think you read some of
the stuff on the wiki... toolkit/base classes)
translate toolkit HEAD branch is the place to look, the underlying
formats are in /translate/storage
there is a separate Pootle branch that I've been using to migrate
towards using the base API (rather than the old PO class API) and
locking, called Pootle-locking-branch
There is a page on that in the wiki too (pootle/locking)
>> I think that we are rolling now, except that, if we want to start
>> with the DB as soon as possible, somebody else -other than Gintautas,
>> who will be busy with the API- should start looking into the design
>> of the DB, and then adapt to the API that Gintautas will develop. It
>> would be great if we can find somebody who has the right expertise
>> and is already involved in Debian.
>>
> I would suggest to wait until the API works with the file-based backend
> before starting work on an RDB binding.
>
> Designing the DB schema should not be difficult here, because we will
> have the object model in the API and we will only need to do a direct
> translation "OO model" -> "RDB schema", which is a mechanical task.
> What is needed here is experience of engineering components that use an
> RDB, and admittedly I don't have much of that.
>
Lets discuss this on IRC as well.
>> information and guidance (types of information and relationships) and
>> Dwayne also has quite a bit of experience on RDB design. The
>> developer would have to work closely with Gintautas, as while doing
>> the DB design, a lot of new necessary pieces of data will come out
>> (metadata that is not in the XLIFF files about workflow, statistics,
>> etc...) and this data will also have to be managed by the API.
>>
> Everything that is in XLIFF will have to be in the API too, because it
> is an abstraction layer. There can't possibly be any piece of
> information that is not abstracted, so there should not be anything new
> directly related to an RDB.
>
One of the issues is that XLIFF being based on XML is hierarchical, and
how best to represent that in a relational database. There's plenty to
talk about here and its not neccessarily straightforward. I agree with
the approach suggested, to separate out a DB backend construction, asd
it enables us to keep moving on the current tasks.
One possibility is to implement a simple DB backend that doesn't have
the complexity of XLIFF... but again, lets discuss this.
>> The API will have to include things that are not implemented now,
>> building on the work that Friedle is doing on statistics, as we will
>> have to keep data on goals, as well as objects that we have not yet
>> implemtented (data on projects, packages..). I would be really happy
>> if we have this at the end of the summer. It will take work from many
>> of us, but will clearly finish the design of the data that will be
>> used by Pootle at the end
>>
> Sure, statistics will be made abstract operations too.
>
>> I agree that this work should take priority over the XML-RPC.
>>
> XML-RPC would be a layer above the new backend, so it makes no sense to
> start doing it now.
>
Agreed. In fact once Pootle interacts with the storage using the Python
API, XML-RPC services could be offered alongside that but Pootle doesn't
*have* to use them (it may just slow things down talking XML-RPC to the
same machine) - although it would be very useful for scalability and
distributing options, so should be supported
>> but can also find others. The translation memory database would be
>> (logically) separate from the DB that includes current projects, as
>> it might come from the outside (from projects external to Pootle) or
>> match translation that are no longer in this Pootle server.
>>
> IOW, the translation memory should be a separate component. Agreed.
>
With translation memory again text indexing is what's really useful...
>> This is probably correct. At the end the editor will have to work
>> with files. In the Pootle server interface, the translator will first
>> identify the files that (s)he wants to work in, and then will decide
>> if (s)he want to translate them online or download them. The editor
>> is called from the server. I would like to have Friedle's or David's
>> opinion on this.
>>
> I'm not Friedle or David ;) but I think that this is not conceptually
> sound. In the online editor the user is not editing a "file", s/he is
> editing translations. That's the whole point: if they want a file,
> they use the "file server" frontend, and if they want to change a single
> string through-the-web, they use a web-based frontend built directly on
> our backend which will have a specific operation for changing a single
> string. Would you agree?
>
The way Pootle currently works you can have it either way - you can be
working project wide or in a specific directory or file, from there you
can do searches or translate untranslated strings etc etc
It is important though that if you are doing string by string operation
the order in the files is preserved as there is a logical continuity often
Sorry I have been dipping in and out of this conversation and have much
more I could say but others are saying plenty :-)
Just for reference must of my work is with relational databases so I do
understand their advantages (hence the pootle/metadata page on the
wiki). Keeping going with working with files directly up till now has
enabled us to handle some of the complexity, which we will need to
continue doing as we move ahead...
Cheers
David
Reply to: